-- Logs begin at Thu 2022-06-23 21:32:14 UTC, end at Thu 2022-06-23 22:54:32 UTC. -- Jun 23 22:21:54.045440 dv-cinder-846173-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 23 22:21:54.053979 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 23 22:21:54.057219 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 23 22:21:54 2022] *** Jun 23 22:21:54.057219 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 23 22:21:54.057219 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 23 22:21:54.057219 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: nodename: dv-cinder-846173-2-iscsi Jun 23 22:21:54.057219 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: machine: x86_64 Jun 23 22:21:54.057219 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: clock source: unix Jun 23 22:21:54.057219 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: pcre jit disabled Jun 23 22:21:54.057219 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: detected number of CPU cores: 4 Jun 23 22:21:54.057219 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: current working directory: / Jun 23 22:21:54.057219 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: detected binary path: /usr/bin/uwsgi-core Jun 23 22:21:54.057219 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: your processes number limit is 47680 Jun 23 22:21:54.057979 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: your memory page size is 4096 bytes Jun 23 22:21:54.057979 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: detected max file descriptor number: 2048 Jun 23 22:21:54.057979 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: lock engine: pthread robust mutexes Jun 23 22:21:54.057979 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: thunder lock: enabled Jun 23 22:21:54.057979 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 23 22:21:54.057979 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 23 22:21:54.101709 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: Python main interpreter initialized at 0x5610cc8c39c0 Jun 23 22:21:54.101709 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: python threads support enabled Jun 23 22:21:54.101709 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: your server socket listen backlog is limited to 100 connections Jun 23 22:21:54.101709 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: your mercy for graceful operations on workers is 90 seconds Jun 23 22:21:54.102801 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: mapped 403077 bytes (393 KB) for 2 cores Jun 23 22:21:54.103215 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: *** Operational MODE: preforking *** Jun 23 22:21:54.103493 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: *** uWSGI is running in multiple interpreter mode *** Jun 23 22:21:54.103493 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: spawned uWSGI master process (pid: 80198) Jun 23 22:21:54.103673 dv-cinder-846173-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 23 22:21:54.108843 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: spawned uWSGI worker 1 (pid: 80201, cores: 1) Jun 23 22:21:54.108843 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: spawned uWSGI worker 2 (pid: 80203, cores: 1) Jun 23 22:21:54.108843 dv-cinder-846173-2-iscsi devstack@keystone.service[80198]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 23 22:21:59.630121 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 22:21:59.630121 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: from cryptography.utils import int_from_bytes Jun 23 22:21:59.630951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 22:21:59.630951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: from cryptography.utils import int_from_bytes Jun 23 22:21:59.631676 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 22:21:59.631676 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: from cryptography.utils import int_from_bytes Jun 23 22:21:59.632134 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 22:21:59.632134 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: from cryptography.utils import int_from_bytes Jun 23 22:21:59.680282 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 22:21:59.680476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 22:21:59.680672 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] command line args: [] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 22:21:59.680948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 22:21:59.681132 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ================================================================================ {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 22:21:59.681317 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 22:21:59.681402 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] admin_token = **** {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.681607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 22:21:59.681691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] config_dir = [] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.681805 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] command line args: [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 22:21:59.681888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.681990 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] config_source = [] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.682094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 22:21:59.682175 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] debug = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.682276 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ================================================================================ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 22:21:59.682352 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.682617 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] admin_token = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.682747 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.682824 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config_dir = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.682917 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.683015 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.683146 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] config_source = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.683248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.683355 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] debug = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.683455 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.683554 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.683684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] list_limit = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.683793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.683892 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] log_config_append = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.683991 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.684083 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.684181 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] log_dir = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.684283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.684378 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] log_file = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.684475 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.684577 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.684689 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.684772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_config_append = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.684892 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.684966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.685040 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.685158 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_dir = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.685237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.685315 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.685436 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.685517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.685587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.685710 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.685786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.685955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.686043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.686202 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.686281 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.686382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.686507 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.686593 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.686697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.686772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.686955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.687037 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.687204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.687298 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.687461 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.687538 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.687703 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.687822 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.687936 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.688182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.688263 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.43/identity {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.688377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] publish_errors = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.688448 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.688606 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.688721 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.688839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.689019 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.689184 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.43/identity {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.689308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.689392 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] publish_errors = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.689490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.689593 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.689729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] standard_threads = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.689821 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.689963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.690045 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.690208 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.690332 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.690405 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.690480 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.690549 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] use_journal = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.690662 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] standard_threads = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.690739 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] use_json = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.690881 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] use_stderr = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.690980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.691083 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] use_syslog = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.691158 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.691318 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.691399 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.691486 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_journal = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.691577 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.691658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_json = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.691815 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_stderr = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.691890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.691991 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] use_syslog = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.692074 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.692223 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 22:21:59.692300 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.692481 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.692562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.692725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.692808 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.692986 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.693067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.693227 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.693307 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.693474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.693560 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.693714 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.693790 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.693963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.694040 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.694198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.694280 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.694444 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.694519 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.694681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.694759 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.694950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.695042 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.695190 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.695277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.695427 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.695988 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.696202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.696477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.696652 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.696725 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.696896 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.696975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.697152 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.697238 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.697396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.697511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.697669 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.697747 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.697918 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.698004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.698155 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.698231 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.698389 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.698466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.698623 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.698705 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.698862 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.698956 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.699137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.699214 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.699387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.699466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.699620 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.699696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.699850 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.699930 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.700084 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.700162 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.700325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.700407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.700562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.700642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.700812 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.700888 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.701042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.701123 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.701277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.701353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.701511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.701586 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.701744 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.701826 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.701980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.702055 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.702204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.702283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.702435 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.702511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.702669 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.702744 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.702906 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.703040 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.703163 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.703239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.703392 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.703467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.703621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.703704 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.703854 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.703935 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.704089 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.704164 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.704335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.704411 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.704565 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.704666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.704800 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.704901 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.705032 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.705135 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.705268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.705370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.705503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.705640 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.705729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.705806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.705957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.706036 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.706195 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.706277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.706434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.706509 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.706669 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.706744 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.706900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.707005 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.707161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.707247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.707395 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.707480 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.707634 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.707709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.707866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.707950 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.708104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.708180 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.708333 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.708417 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.708596 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.708672 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.708833 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.708915 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.709064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.709140 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.709301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.709382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.709535 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.709611 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.709771 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.709851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.710023 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.710110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.710266 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.710343 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.710507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.710587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.710745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.710829 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.711003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.711082 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.711945 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.712122 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.712319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.712405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.712559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.712643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.712826 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.712903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.713071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.713156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.713316 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.713394 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.713559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.713636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.713800 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.713883 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.714036 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.714118 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.714275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.714351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.714511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.714661 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.714735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.714805 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.714989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.715070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.715226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.715314 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.715464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.715556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.715697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.715775 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.715941 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.716017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.716179 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.716267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.716416 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.716497 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.716650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.716727 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.716883 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.716967 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.717115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.717197 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.717352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.717428 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.717584 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.717661 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] auth.external = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.717826 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.717902 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.718074 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.718150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.718324 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.718402 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.718561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] auth.password = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.718642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.718796 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] auth.token = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.718881 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.719063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.719175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.719305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.719392 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.719551 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.external = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.719628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.719782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.719863 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.720033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.720112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.720282 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.720360 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.720514 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.password = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.720590 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.720746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] auth.token = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.720825 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] credential.caching = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.720989 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.721068 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.721227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.721304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.721474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.721553 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.721707 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.721784 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.721951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.722027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.722193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.722272 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.722421 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.722497 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.722661 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.722747 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.722912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.723046 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.723184 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.723262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.723432 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.723510 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.723671 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.723750 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.723915 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.723993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.724153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.724230 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.724419 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.724498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.724667 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] federation.caching = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.724744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.724901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.724977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.725147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.725225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.725397 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.725474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.725631 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.725707 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.725881 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.725960 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.726120 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.726198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.726377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.726454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.726609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.726687 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.726907 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.727171 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.727335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.727415 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.727598 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.727677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] identity.caching = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.727850 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.727928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.728085 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.728163 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.728335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.728415 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.728572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.728649 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.728818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.728971 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.729046 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.729117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.729277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.729355 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.729518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.729602 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.729764 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.729842 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.729994 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.730073 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.730229 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.730313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.730461 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.730539 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.730693 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.730770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.730995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.731096 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.731202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.731279 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.731439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.731517 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.731679 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.731764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.731927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.732005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.732175 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.732252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.732406 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.732490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.732638 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.732715 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.732878 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.732956 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.733112 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.733196 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.733351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.733429 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.733596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.733673 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.733839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.733923 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.734085 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.734162 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.734321 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.734397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.734571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.734656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.734807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.734884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.735066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.735154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.735306 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.735391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.735538 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.735616 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.735769 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.735846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.736011 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.736097 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.736247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.736325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.736487 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.736565 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.736715 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.736800 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.736957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.737035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.737199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.737277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.737432 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.737516 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.737677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.737755 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.737923 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.738001 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.738153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.738239 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.738384 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.738461 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.738612 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.738688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.738850 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.738964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.739114 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.739192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.739350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.739426 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.739585 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.739674 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.739814 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.739891 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.740058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.740137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.740302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.740387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.740534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.740613 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.740764 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.740841 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.741011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.741090 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.741255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.741331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.741487 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.741564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.741717 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.741795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.741948 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.742025 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.742179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.742257 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.742440 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.742526 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.742672 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.742936 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.743176 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.743255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.743420 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.743499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.743668 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.743746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.743914 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.743991 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.744144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.744221 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.744387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.744467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.744624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.744701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.744860 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.744938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.745096 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.745173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.745330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.745407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.745562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.745641 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.745809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.745886 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.746049 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.746126 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.746292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.746371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.746521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.746598 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.746768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.746846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.747020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.747101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.747266 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.747343 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.747503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.747590 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.747741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.747821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.747978 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.748056 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.748211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.748287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.748451 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.748529 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.748694 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] resource.caching = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.748771 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.748930 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.749006 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.749180 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.749257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.749408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.749485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.749655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.749732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.749898 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.749977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.750138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.750220 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.750376 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.750453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.750614 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.750691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.750853 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.750946 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.751113 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] role.caching = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.751191 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.751350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.751429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] role.driver = sql {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.751581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.751657 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.751829 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.751905 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.752077 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.752156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.752310 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.752387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.752559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.752636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.752807 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.752887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.753040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.753117 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.753288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.753366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.753530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.753609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.753759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.753835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.753991 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.754075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.754238 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.754317 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.754470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.754547 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.754716 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.754800 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.754983 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.755146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.755224 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.755388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.755466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.755642 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.755723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.756053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.756135 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.756208 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.756278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.756358 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.756431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.756590 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.756669 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.756845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.756923 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.757096 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.757175 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.757341 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.757418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.757578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.757656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.757814 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.758484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.758587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.758803 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.759085 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.759204 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.759338 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.759464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.759579 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.759721 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.759819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.759975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.760061 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.760228 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.760307 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] token.caching = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.760465 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.760543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.760723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.760808 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.760978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.761054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.761218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.761295 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.761455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.761534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.761689 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.761767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.761923 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.762001 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.762169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.762247 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.762419 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.762497 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.762661 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.762739 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.762897 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.762995 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.763158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.763236 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.763386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.763464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.763625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.763704 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.763862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.763950 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.764106 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.764183 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.764340 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.764418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.764597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.764675 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.connection = **** {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.764848 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.764935 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.765087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.765165 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.765332 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.765411 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.765576 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.765654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.765812 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.765888 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.766051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.766129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.766287 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.766363 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.766525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.766601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.766770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.766847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.767033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.767115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.767262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.767340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.767506 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.767585 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.767754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.767830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.767995 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.768081 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.768242 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.768337 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.768475 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.768553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.768717 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.768795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.768953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.769038 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.769111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80201) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 22:21:59.769182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.769411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.769647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.769887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.770136 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.770375 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.770621 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.770860 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.771136 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.771375 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.771876 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.772107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.772349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.772586 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 22:21:59.772758 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80203) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 22:21:59.799998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:21:59.800231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:21:59.800465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:21:59.803495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:21:59.803733 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:21:59.803957 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:21:59.809150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:21:59.813002 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:21:59.838362 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:21:59.838598 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:21:59.840670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:21:59.840909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:21:59.981304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:21:59.985463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.030327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.030563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.032073 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.032313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.095137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.095373 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.095548 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.095660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.095784 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.095871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.096009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.096093 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.096227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.096318 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.096447 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.096664 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.121681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.122200 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.122200 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.122372 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.122438 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.122500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.122561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.122630 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.122779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.122859 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.123027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.123132 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.123291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.123510 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.135232 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.135825 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.148139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.148377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.148474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.148632 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.148819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.149014 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.149346 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.149583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.149671 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.149816 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.149904 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80201) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.150042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.150262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.150478 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80203) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 22:22:00.167485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.167700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 22:22:00.168041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.168143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.168253 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 22:22:00.168378 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 22:22:00.168525 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.168669 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.168787 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.168885 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 22:22:00.169046 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.169128 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.169302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.169383 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.169622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.169882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.180937 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.181212 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.181481 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.181766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.181865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.182052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.182319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.182598 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.190985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.191120 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.194261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.194542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.194826 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.195140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.195425 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.196229 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.196517 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.196763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.196906 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.197033 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.197196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.197313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.197482 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.197589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.197750 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.197894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.198020 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.198140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.198295 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.198566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.198842 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.199138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.225219 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.225951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.226947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.227594 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.227680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.227929 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.232926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.233246 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.233333 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.233539 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.233647 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.233934 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.240845 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.241136 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.241303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.241587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.246648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.246748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.246956 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.247058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.247249 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.247333 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.247539 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.247624 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.247817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.247909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.248093 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.248177 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.248363 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.248445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.248627 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.248712 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.268285 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.268385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.268575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.268674 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.273451 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.273631 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.277768 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.278167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.278273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.278618 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.283240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.283767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.286860 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.287437 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.291106 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.291232 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.291402 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.291533 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.291687 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.291823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.291976 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.292112 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.292253 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.292410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.292534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.292695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.312450 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.312671 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.312754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.312955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.313036 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.313224 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.313305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.313503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.321758 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.322471 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.323498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.323782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.324225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.324504 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.327021 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.327321 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.327629 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.327738 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.327939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.328041 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.328282 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.328364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.328566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.328848 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.341633 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.341912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.342202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.342485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.342640 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.342954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.343239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.343524 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.344106 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.344409 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.344708 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.345144 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.345438 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.345733 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.362772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.364641 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.366128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.369144 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.370028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.371798 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.372503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.373469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.373751 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.374037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.374235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.375909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.376189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.376475 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.381421 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.383607 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.384838 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.386449 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.387000 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.387296 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.387694 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.387979 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.389707 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.390011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.390406 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.390687 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.397422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.397708 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.398297 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.400406 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.400692 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.400974 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.406285 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80201) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.406571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.406855 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.407158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.407432 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.407812 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.408108 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.408382 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.408657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.408895 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80203) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 22:22:00.409029 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.409155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.409291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.409420 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.409559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.409698 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.409874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.410004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.410130 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80201) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.410320 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.410600 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.410887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.411199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.411482 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.411762 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.412027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.412302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.412579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80203) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 22:22:00.450383 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5610cc8c39c0 pid: 80201 (default app) Jun 23 22:22:00.453696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5610cc8c39c0 pid: 80203 (default app) Jun 23 22:22:00.453852 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547a55e-3b1e-41ec-a869-665a944fc305 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:00.454153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547a55e-3b1e-41ec-a869-665a944fc305 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:00.454442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547a55e-3b1e-41ec-a869-665a944fc305 None None] PATH_INFO: `/v3` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:00.458628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1/1] 10.222.0.43 () {50 vars in 791 bytes} [Thu Jun 23 22:22:00 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:08.075477 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660535ac-5384-4b36-b9e4-b6a1599c0206 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:08.076061 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660535ac-5384-4b36-b9e4-b6a1599c0206 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:08.076061 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660535ac-5384-4b36-b9e4-b6a1599c0206 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:08.080334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1/2] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:22:08.085835 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493424d-98fe-4cc2-8cae-714a61968a23 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:08.086258 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493424d-98fe-4cc2-8cae-714a61968a23 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:08.086407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493424d-98fe-4cc2-8cae-714a61968a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:08.156293 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: INFO dbcounter [None req-1493424d-98fe-4cc2-8cae-714a61968a23 None None] Registered counter for database keystone Jun 23 22:22:08.177053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG oslo_db.sqlalchemy.engines [None req-1493424d-98fe-4cc2-8cae-714a61968a23 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80201) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 22:22:08.183270 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writer thread running {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 22:22:08.360004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG passlib.handlers.bcrypt [None req-1493424d-98fe-4cc2-8cae-714a61968a23 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80201) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 22:22:08.360323 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG passlib.handlers.bcrypt [None req-1493424d-98fe-4cc2-8cae-714a61968a23 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80201) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 22:22:08.414645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-1493424d-98fe-4cc2-8cae-714a61968a23 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:08.416247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1493424d-98fe-4cc2-8cae-714a61968a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:08.537855 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 2/3] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 22:22:08.688741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660535ac-5384-4b36-b9e4-b6a1599c0206 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:08.758129 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: INFO dbcounter [None req-660535ac-5384-4b36-b9e4-b6a1599c0206 None None] Registered counter for database keystone Jun 23 22:22:08.777290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG oslo_db.sqlalchemy.engines [None req-660535ac-5384-4b36-b9e4-b6a1599c0206 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80203) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 22:22:08.783452 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writer thread running {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 22:22:08.903512 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660535ac-5384-4b36-b9e4-b6a1599c0206 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:08.907756 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808bd20-c643-4fef-913e-0920939023a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:08.908415 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808bd20-c643-4fef-913e-0920939023a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:08.908704 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808bd20-c643-4fef-913e-0920939023a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:08.908984 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808bd20-c643-4fef-913e-0920939023a3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:08.917770 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2808bd20-c643-4fef-913e-0920939023a3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:08.969646 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2808bd20-c643-4fef-913e-0920939023a3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 22:22:08.969933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2808bd20-c643-4fef-913e-0920939023a3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:08.980337 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 2/4] 10.222.0.43 () {58 vars in 1324 bytes} [Thu Jun 23 22:22:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 22:22:08.986275 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493424d-98fe-4cc2-8cae-714a61968a23 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:09.001094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493424d-98fe-4cc2-8cae-714a61968a23 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:09.005466 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f30e69-8672-42c7-a1c5-4bd3716e8376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:09.006573 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f30e69-8672-42c7-a1c5-4bd3716e8376 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:09.006865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f30e69-8672-42c7-a1c5-4bd3716e8376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:09.007190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f30e69-8672-42c7-a1c5-4bd3716e8376 admin admin] PATH_INFO: `/v3/projects/1aef1dab34da4006bf2529ee1b5120da` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:09.011619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f30e69-8672-42c7-a1c5-4bd3716e8376 admin admin] RBAC: Authorizing `identity:get_project(project_id=1aef1dab34da4006bf2529ee1b5120da)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:09.062201 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b5f30e69-8672-42c7-a1c5-4bd3716e8376 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 22:22:09.062540 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f30e69-8672-42c7-a1c5-4bd3716e8376 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:09.067109 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 3/5] 10.222.0.43 () {56 vars in 1207 bytes} [Thu Jun 23 22:22:08 2022] GET /identity/v3/projects/1aef1dab34da4006bf2529ee1b5120da => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:11.099824 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc1627-62f1-469a-ab87-038a2b15a7cd None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:11.100361 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc1627-62f1-469a-ab87-038a2b15a7cd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:11.100458 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc1627-62f1-469a-ab87-038a2b15a7cd None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:11.105697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 3/6] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:11.111080 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f3bbf-0ea6-41e6-a355-9d22fb342d53 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:11.111443 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f3bbf-0ea6-41e6-a355-9d22fb342d53 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:11.111658 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f3bbf-0ea6-41e6-a355-9d22fb342d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:11.131664 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-481f3bbf-0ea6-41e6-a355-9d22fb342d53 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:11.133852 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-481f3bbf-0ea6-41e6-a355-9d22fb342d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:11.141828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 4/7] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 22:22:11.164839 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc1627-62f1-469a-ab87-038a2b15a7cd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:11.177074 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc1627-62f1-469a-ab87-038a2b15a7cd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:11.181471 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1963e3-4ecf-4297-9a72-20537761029f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:11.182105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1963e3-4ecf-4297-9a72-20537761029f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:11.182393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1963e3-4ecf-4297-9a72-20537761029f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:11.182673 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1963e3-4ecf-4297-9a72-20537761029f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:11.190235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1963e3-4ecf-4297-9a72-20537761029f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:11.192028 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1963e3-4ecf-4297-9a72-20537761029f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:11.201300 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 4/8] 10.222.0.43 () {58 vars in 1324 bytes} [Thu Jun 23 22:22:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:22:11.206627 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f3bbf-0ea6-41e6-a355-9d22fb342d53 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:11.216088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f3bbf-0ea6-41e6-a355-9d22fb342d53 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:11.220465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c6625a-c1fb-40a1-ad94-d1683011a2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:11.221333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6625a-c1fb-40a1-ad94-d1683011a2e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:11.221627 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6625a-c1fb-40a1-ad94-d1683011a2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:11.221912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6625a-c1fb-40a1-ad94-d1683011a2e2 admin admin] PATH_INFO: `/v3/users/c3c5fce3a5074734811c3407aac7fd6b` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:11.223358 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6625a-c1fb-40a1-ad94-d1683011a2e2 admin admin] RBAC: Authorizing `identity:get_user(user_id=c3c5fce3a5074734811c3407aac7fd6b)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:11.225183 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6625a-c1fb-40a1-ad94-d1683011a2e2 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:11.226738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 5/9] 10.222.0.43 () {56 vars in 1198 bytes} [Thu Jun 23 22:22:11 2022] GET /identity/v3/users/c3c5fce3a5074734811c3407aac7fd6b => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:13.291814 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cb44c-3f40-4712-80fe-246e1abe0e12 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:13.292345 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cb44c-3f40-4712-80fe-246e1abe0e12 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:13.292345 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cb44c-3f40-4712-80fe-246e1abe0e12 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:13.293709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 5/10] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:13.298834 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38591f62-1eda-4c8e-ac15-1c4e4b2eeb5d None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:13.299271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38591f62-1eda-4c8e-ac15-1c4e4b2eeb5d None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:13.299557 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38591f62-1eda-4c8e-ac15-1c4e4b2eeb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:13.316556 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-38591f62-1eda-4c8e-ac15-1c4e4b2eeb5d None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:13.317823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-38591f62-1eda-4c8e-ac15-1c4e4b2eeb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:13.321795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da0cea-5f40-43be-9a9c-d4278106637e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:13.322088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da0cea-5f40-43be-9a9c-d4278106637e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:13.322373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da0cea-5f40-43be-9a9c-d4278106637e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:13.323755 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 6/11] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:22:13.328838 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5c1dd-2332-4fa5-a207-a1bad8f08569 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:13.329298 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5c1dd-2332-4fa5-a207-a1bad8f08569 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:13.329580 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5c1dd-2332-4fa5-a207-a1bad8f08569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:13.368710 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG passlib.handlers.bcrypt [None req-bbb5c1dd-2332-4fa5-a207-a1bad8f08569 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80203) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 22:22:13.369136 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG passlib.handlers.bcrypt [None req-bbb5c1dd-2332-4fa5-a207-a1bad8f08569 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80203) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 22:22:13.401803 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 6/12] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:13.415683 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bbb5c1dd-2332-4fa5-a207-a1bad8f08569 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:13.417279 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bbb5c1dd-2332-4fa5-a207-a1bad8f08569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:13.424448 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38591f62-1eda-4c8e-ac15-1c4e4b2eeb5d None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:13.448323 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:13.517331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 7/13] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:22:13.528751 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38591f62-1eda-4c8e-ac15-1c4e4b2eeb5d None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:13.539879 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5c1dd-2332-4fa5-a207-a1bad8f08569 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:13.563299 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef9a14-cfd5-4322-99a9-0ec30bf87e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:13.564107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef9a14-cfd5-4322-99a9-0ec30bf87e4e None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:13.564402 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef9a14-cfd5-4322-99a9-0ec30bf87e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:13.564682 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef9a14-cfd5-4322-99a9-0ec30bf87e4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:13.575724 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef9a14-cfd5-4322-99a9-0ec30bf87e4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:13.587336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-22ef9a14-cfd5-4322-99a9-0ec30bf87e4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:22:13.587336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:13.587336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:13.587336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:13.587336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:13.587336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:13.587336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:13.587336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:13.587336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:13.587336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:13.587336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:13.587336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:13.587336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:13.587336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:13.587336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:13.587336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:13.588273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:13.588273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:13.588273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:13.588273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:13.588273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:13.588273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:13.588273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:13.588273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:13.588273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:13.588273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:13.588273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:13.588273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:13.588273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:13.588273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:13.588273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:13.588273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:13.589129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:13.589129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:13.589129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:13.589129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:13.589129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:13.589129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:13.589129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:13.589129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:13.589129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:13.589129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:13.589129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:22:13.589129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:13.589129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 7/14] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:22:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:22:13.594211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef9a14-cfd5-4322-99a9-0ec30bf87e4e None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:13.641700 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5c1dd-2332-4fa5-a207-a1bad8f08569 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:13.644411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:13.678547 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:13.683309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555b6e2-dbbd-4324-ba14-f1b13bda0412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:13.684220 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555b6e2-dbbd-4324-ba14-f1b13bda0412 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:13.684517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555b6e2-dbbd-4324-ba14-f1b13bda0412 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:13.684805 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555b6e2-dbbd-4324-ba14-f1b13bda0412 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:13.697598 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef9a14-cfd5-4322-99a9-0ec30bf87e4e None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:13.731341 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cd36b-c7ef-453b-9d8b-5fd2c72c3791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:13.732007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cd36b-c7ef-453b-9d8b-5fd2c72c3791 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:13.732292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cd36b-c7ef-453b-9d8b-5fd2c72c3791 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:13.732575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cd36b-c7ef-453b-9d8b-5fd2c72c3791 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:13.733327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134cd36b-c7ef-453b-9d8b-5fd2c72c3791 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:13.733646 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134cd36b-c7ef-453b-9d8b-5fd2c72c3791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:13.735069 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134cd36b-c7ef-453b-9d8b-5fd2c72c3791 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:13.746890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 8/15] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:13.752283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cd36b-c7ef-453b-9d8b-5fd2c72c3791 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:13.752671 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555b6e2-dbbd-4324-ba14-f1b13bda0412 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:13.754195 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555b6e2-dbbd-4324-ba14-f1b13bda0412 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:13.852478 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cd36b-c7ef-453b-9d8b-5fd2c72c3791 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:13.856134 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 8/16] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 22:22:13.861970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555b6e2-dbbd-4324-ba14-f1b13bda0412 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:13.864695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:13.887084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4c67f-54e8-46af-83ca-4307878a2f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:13.887972 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4c67f-54e8-46af-83ca-4307878a2f26 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:13.888264 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4c67f-54e8-46af-83ca-4307878a2f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:13.888549 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4c67f-54e8-46af-83ca-4307878a2f26 None admin] PATH_INFO: `/v3/users/c3c5fce3a5074734811c3407aac7fd6b` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:13.889754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4c67f-54e8-46af-83ca-4307878a2f26 None admin] RBAC: Authorizing `identity:get_user(user_id=c3c5fce3a5074734811c3407aac7fd6b)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:13.891289 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4c67f-54e8-46af-83ca-4307878a2f26 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:13.892927 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 9/17] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:13 2022] GET /identity/v3/users/c3c5fce3a5074734811c3407aac7fd6b => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:13.898673 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4c67f-54e8-46af-83ca-4307878a2f26 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:13.912592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:13.960766 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555b6e2-dbbd-4324-ba14-f1b13bda0412 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:13.996025 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18b1f9-97ab-41f6-89c0-5d30f4c13b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:13.996823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18b1f9-97ab-41f6-89c0-5d30f4c13b8b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:13.997144 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18b1f9-97ab-41f6-89c0-5d30f4c13b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:13.997431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18b1f9-97ab-41f6-89c0-5d30f4c13b8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:13.999404 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4c67f-54e8-46af-83ca-4307878a2f26 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:14.016482 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18b1f9-97ab-41f6-89c0-5d30f4c13b8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:14.017958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18b1f9-97ab-41f6-89c0-5d30f4c13b8b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:14.034566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc12e3-c8e6-4e69-8c86-761e761fbef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:14.035402 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc12e3-c8e6-4e69-8c86-761e761fbef5 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:14.035695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc12e3-c8e6-4e69-8c86-761e761fbef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:14.035979 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc12e3-c8e6-4e69-8c86-761e761fbef5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:14.037170 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc12e3-c8e6-4e69-8c86-761e761fbef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:14.038278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-0a18b1f9-97ab-41f6-89c0-5d30f4c13b8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:22:14.038278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:14.038278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:14.038278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:14.038278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:14.038278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:14.038278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:14.038278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:14.038278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:14.038278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:14.038278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:14.038278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:14.038278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:22:14.038278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:22:14.038278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:22:14.039677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:22:14.039677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:14.039677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:14.039677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:14.039677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:14.039677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:14.039677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:14.039677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:14.039677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:14.039677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:14.039677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:14.039677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:14.039677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:14.039677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:14.039677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:14.039677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:14.039677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:14.045796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:22:14.045796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:22:14.045796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:22:14.045796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:22:14.046069 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc12e3-c8e6-4e69-8c86-761e761fbef5 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:14.046156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 9/18] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:22:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:22:14.046233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 10/19] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:22:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:14.046326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18b1f9-97ab-41f6-89c0-5d30f4c13b8b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:14.046869 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc12e3-c8e6-4e69-8c86-761e761fbef5 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:14.103155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:14.128603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:14.147166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc12e3-c8e6-4e69-8c86-761e761fbef5 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:14.152448 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18b1f9-97ab-41f6-89c0-5d30f4c13b8b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:14.181373 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe57dd8-47bb-4504-83cb-264c11a1cc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:14.182042 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe57dd8-47bb-4504-83cb-264c11a1cc00 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:14.182330 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe57dd8-47bb-4504-83cb-264c11a1cc00 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:14.182616 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe57dd8-47bb-4504-83cb-264c11a1cc00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:14.183499 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe57dd8-47bb-4504-83cb-264c11a1cc00 None admin] RBAC: Adding query filter params (role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.domain.id=default, user.id=c3c5fce3a5074734811c3407aac7fd6b) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:14.183801 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe57dd8-47bb-4504-83cb-264c11a1cc00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:14.185252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe57dd8-47bb-4504-83cb-264c11a1cc00 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:14.185359 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f82fcd-b68a-406d-9543-ace300d22d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:14.186004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f82fcd-b68a-406d-9543-ace300d22d4f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:14.186300 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f82fcd-b68a-406d-9543-ace300d22d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:14.186590 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f82fcd-b68a-406d-9543-ace300d22d4f None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:14.187356 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f82fcd-b68a-406d-9543-ace300d22d4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:14.187672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f82fcd-b68a-406d-9543-ace300d22d4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:14.189052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f82fcd-b68a-406d-9543-ace300d22d4f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:14.191941 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 11/20] 10.222.0.43 () {56 vars in 1322 bytes} [Thu Jun 23 22:22:14 2022] GET /identity/v3/role_assignments?user.id=c3c5fce3a5074734811c3407aac7fd6b&scope.domain.id=default&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:14.208200 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 10/21] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:16.446054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04990866-7eb5-425d-9558-2bbb72096ef7 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:16.447090 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04990866-7eb5-425d-9558-2bbb72096ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:16.447687 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04990866-7eb5-425d-9558-2bbb72096ef7 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:16.449289 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 12/22] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:16.454958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abce2f1-deca-44ed-b39c-3351484909fc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:16.455406 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abce2f1-deca-44ed-b39c-3351484909fc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:16.455526 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abce2f1-deca-44ed-b39c-3351484909fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:16.460229 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d707792-dffe-4193-801c-b3f3df7d603f None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:16.460801 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d707792-dffe-4193-801c-b3f3df7d603f None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:16.461308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d707792-dffe-4193-801c-b3f3df7d603f None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:16.463175 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 13/23] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:16.468148 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d9afd-84b6-4609-a78b-ec13b522d471 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:16.468686 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d9afd-84b6-4609-a78b-ec13b522d471 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:16.469193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d9afd-84b6-4609-a78b-ec13b522d471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:16.483507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8abce2f1-deca-44ed-b39c-3351484909fc None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:16.484918 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8abce2f1-deca-44ed-b39c-3351484909fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:16.500015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-2d0d9afd-84b6-4609-a78b-ec13b522d471 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:16.501372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2d0d9afd-84b6-4609-a78b-ec13b522d471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:16.629462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 14/24] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:16.649982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 11/25] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:16.681803 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abce2f1-deca-44ed-b39c-3351484909fc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:16.682001 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d9afd-84b6-4609-a78b-ec13b522d471 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:16.775330 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:16.800090 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:16.817016 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d9afd-84b6-4609-a78b-ec13b522d471 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:16.882370 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee73cef-0341-48ea-b204-d16fff37d79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:16.883828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee73cef-0341-48ea-b204-d16fff37d79c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:16.884315 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee73cef-0341-48ea-b204-d16fff37d79c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:16.884854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee73cef-0341-48ea-b204-d16fff37d79c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:16.888253 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abce2f1-deca-44ed-b39c-3351484909fc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:16.903555 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee73cef-0341-48ea-b204-d16fff37d79c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:16.917313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-cee73cef-0341-48ea-b204-d16fff37d79c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:22:16.917313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:16.917313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:16.917313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:16.917313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:16.917313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:16.917313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:16.917313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:16.917313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:16.917313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:16.917313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:16.917313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:16.917313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:16.917313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:16.917313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:16.917313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:16.918202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:16.918202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:16.918202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:16.918202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:16.918202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:16.918202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:16.918202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:16.918202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:16.918202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:16.918202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:16.918202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:16.918202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:16.918202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:16.918202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:16.918202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:16.918202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:16.918961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:16.918961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:16.918961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:16.918961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:16.918961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:16.918961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:16.918961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:16.918961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:16.918961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:16.918961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:16.918961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:22:16.918961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:16.918961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 15/26] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:22:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:16.925133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee73cef-0341-48ea-b204-d16fff37d79c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:16.943776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd308e6-5339-4210-b6bf-89851188eb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:16.944918 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd308e6-5339-4210-b6bf-89851188eb11 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:16.946209 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd308e6-5339-4210-b6bf-89851188eb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:16.946687 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd308e6-5339-4210-b6bf-89851188eb11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:16.961274 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd308e6-5339-4210-b6bf-89851188eb11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:16.967839 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd308e6-5339-4210-b6bf-89851188eb11 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:17.001852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-fbd308e6-5339-4210-b6bf-89851188eb11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:22:17.001852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:17.001852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:17.001852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:17.001852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:17.001852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:17.001852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:17.001852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:17.001852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:17.001852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:17.001852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:17.001852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:17.001852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:22:17.001852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:22:17.001852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:22:17.002749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:22:17.002749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:17.002749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:17.002749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:17.002749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:17.002749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:17.002749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:17.002749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:17.002749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:17.002749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:17.002749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:17.002749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:17.002749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:17.002749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:17.002749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:17.002749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:17.002749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:17.003583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:22:17.003583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:22:17.003583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:22:17.003583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:22:17.005590 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 12/27] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:22:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:17.024340 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd308e6-5339-4210-b6bf-89851188eb11 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:17.079554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee73cef-0341-48ea-b204-d16fff37d79c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:17.127674 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57621633-8557-42ca-b1ed-303249066854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:17.128373 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57621633-8557-42ca-b1ed-303249066854 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:17.128660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57621633-8557-42ca-b1ed-303249066854 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:17.128944 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57621633-8557-42ca-b1ed-303249066854 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:17.129716 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57621633-8557-42ca-b1ed-303249066854 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:17.130030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57621633-8557-42ca-b1ed-303249066854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:17.138955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57621633-8557-42ca-b1ed-303249066854 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:17.138955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:17.143793 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 16/28] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:17.145359 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911dc7e2-ed16-4e1c-b946-590065e78291 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:17.145791 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911dc7e2-ed16-4e1c-b946-590065e78291 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:17.146219 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911dc7e2-ed16-4e1c-b946-590065e78291 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:17.147887 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 17/29] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:17.149055 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911dc7e2-ed16-4e1c-b946-590065e78291 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:17.239289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:17.246350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd308e6-5339-4210-b6bf-89851188eb11 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:17.300459 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6d24d-ef34-4282-a904-a5849707d444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:17.301125 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6d24d-ef34-4282-a904-a5849707d444 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:17.301577 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6d24d-ef34-4282-a904-a5849707d444 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:17.301997 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6d24d-ef34-4282-a904-a5849707d444 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:17.302881 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6d24d-ef34-4282-a904-a5849707d444 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:17.303361 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6d24d-ef34-4282-a904-a5849707d444 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:17.304963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6d24d-ef34-4282-a904-a5849707d444 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:17.316693 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 13/30] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:17.318197 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b504b68-27a7-466e-8d8b-5b74421ea42b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:17.318640 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b504b68-27a7-466e-8d8b-5b74421ea42b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:17.319098 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b504b68-27a7-466e-8d8b-5b74421ea42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:17.330054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911dc7e2-ed16-4e1c-b946-590065e78291 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:17.337546 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0b504b68-27a7-466e-8d8b-5b74421ea42b None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:17.339166 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0b504b68-27a7-466e-8d8b-5b74421ea42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:17.369520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478b626-0a60-4d9f-b038-71e38f5a5e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:17.370895 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478b626-0a60-4d9f-b038-71e38f5a5e16 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:17.371217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478b626-0a60-4d9f-b038-71e38f5a5e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:17.371502 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478b626-0a60-4d9f-b038-71e38f5a5e16 None admin] PATH_INFO: `/v3/users/c3c5fce3a5074734811c3407aac7fd6b` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:17.372756 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478b626-0a60-4d9f-b038-71e38f5a5e16 None admin] RBAC: Authorizing `identity:get_user(user_id=c3c5fce3a5074734811c3407aac7fd6b)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:17.374437 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478b626-0a60-4d9f-b038-71e38f5a5e16 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:17.376279 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 18/31] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:17 2022] GET /identity/v3/users/c3c5fce3a5074734811c3407aac7fd6b => generated 247 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:17.380182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc142f3-dcc4-460f-8395-1c1d04ab5568 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:17.381488 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc142f3-dcc4-460f-8395-1c1d04ab5568 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:17.383341 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc142f3-dcc4-460f-8395-1c1d04ab5568 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:17.385793 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 19/32] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:17.386655 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc142f3-dcc4-460f-8395-1c1d04ab5568 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:17.435097 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 14/33] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:17.437289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b504b68-27a7-466e-8d8b-5b74421ea42b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:17.484017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:17.544816 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc142f3-dcc4-460f-8395-1c1d04ab5568 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:17.547530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:17.561962 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b504b68-27a7-466e-8d8b-5b74421ea42b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:17.594188 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9a206-800b-4cd9-a3a2-4580bc0d610f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:17.594787 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9a206-800b-4cd9-a3a2-4580bc0d610f None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:17.595105 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9a206-800b-4cd9-a3a2-4580bc0d610f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:17.595770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9a206-800b-4cd9-a3a2-4580bc0d610f None admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:17.596659 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9a206-800b-4cd9-a3a2-4580bc0d610f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:17.598210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9a206-800b-4cd9-a3a2-4580bc0d610f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:17.600883 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1bd1a1-6127-41e8-917c-83dade8674d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:17.602714 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bd1a1-6127-41e8-917c-83dade8674d0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:17.603249 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bd1a1-6127-41e8-917c-83dade8674d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:17.603719 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bd1a1-6127-41e8-917c-83dade8674d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:17.605401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1bd1a1-6127-41e8-917c-83dade8674d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:17.607145 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1bd1a1-6127-41e8-917c-83dade8674d0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:17.609247 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 15/34] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:22:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:17.610809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb18ce-1a42-446e-a6f5-318021f0edb8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:17.611323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb18ce-1a42-446e-a6f5-318021f0edb8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:17.611796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb18ce-1a42-446e-a6f5-318021f0edb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:17.630625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7dcb18ce-1a42-446e-a6f5-318021f0edb8 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:17.632413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7dcb18ce-1a42-446e-a6f5-318021f0edb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:17.714951 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 16/35] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:17.714951 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb18ce-1a42-446e-a6f5-318021f0edb8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:17.782537 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 20/36] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 22:22:17 2022] POST /identity/v3/projects => generated 301 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:22:17.783693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9a206-800b-4cd9-a3a2-4580bc0d610f None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:17.818871 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:17.839580 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb18ce-1a42-446e-a6f5-318021f0edb8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:17.877523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1906b-af4c-45d7-8e27-44d2732fc653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:17.878528 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1906b-af4c-45d7-8e27-44d2732fc653 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:17.879041 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1906b-af4c-45d7-8e27-44d2732fc653 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:17.879550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1906b-af4c-45d7-8e27-44d2732fc653 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:17.880588 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1906b-af4c-45d7-8e27-44d2732fc653 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:17.882290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1906b-af4c-45d7-8e27-44d2732fc653 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:17.941656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9a206-800b-4cd9-a3a2-4580bc0d610f None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:17.972762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:18.016321 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f4542-30d4-4956-98c1-5e6389125eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:18.017240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f4542-30d4-4956-98c1-5e6389125eac None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:18.017535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f4542-30d4-4956-98c1-5e6389125eac None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:18.017826 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f4542-30d4-4956-98c1-5e6389125eac None admin] PATH_INFO: `/v3/domains/default/users/c3c5fce3a5074734811c3407aac7fd6b/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:18.025036 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f4542-30d4-4956-98c1-5e6389125eac None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c3c5fce3a5074734811c3407aac7fd6b, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:18.026672 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f4542-30d4-4956-98c1-5e6389125eac None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:18.032470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 17/37] 10.222.0.43 () {60 vars in 1131 bytes} [Thu Jun 23 22:22:17 2022] POST /identity/v3/roles => generated 219 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:22:18.033941 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1906b-af4c-45d7-8e27-44d2732fc653 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:18.059226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 21/38] 10.222.0.43 () {58 vars in 1361 bytes} [Thu Jun 23 22:22:17 2022] PUT /identity/v3/domains/default/users/c3c5fce3a5074734811c3407aac7fd6b/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:22:18.065904 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50ab88-a680-4f02-b6c1-9018bc07a33d None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:18.066256 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50ab88-a680-4f02-b6c1-9018bc07a33d None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:18.066503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50ab88-a680-4f02-b6c1-9018bc07a33d None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:18.068067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 22/39] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:22:18.076562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d325ce-89fa-4fe4-bc6d-57411f7f8683 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:18.076906 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d325ce-89fa-4fe4-bc6d-57411f7f8683 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:18.077161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d325ce-89fa-4fe4-bc6d-57411f7f8683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:18.111490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-44d325ce-89fa-4fe4-bc6d-57411f7f8683 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:18.113177 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-44d325ce-89fa-4fe4-bc6d-57411f7f8683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:18.177813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1906b-af4c-45d7-8e27-44d2732fc653 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:18.221836 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774622b3-f663-4505-b28a-b3a4996ef229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:18.223037 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774622b3-f663-4505-b28a-b3a4996ef229 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:18.223552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774622b3-f663-4505-b28a-b3a4996ef229 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:18.224046 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774622b3-f663-4505-b28a-b3a4996ef229 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:18.225024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774622b3-f663-4505-b28a-b3a4996ef229 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:18.227824 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774622b3-f663-4505-b28a-b3a4996ef229 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:18.249030 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 18/40] 10.222.0.43 () {60 vars in 1131 bytes} [Thu Jun 23 22:22:18 2022] POST /identity/v3/roles => generated 225 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:22:18.288707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 23/41] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:22:18.315435 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774622b3-f663-4505-b28a-b3a4996ef229 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:18.325686 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:18.337528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d55ed-0c5a-4220-8069-183faa33f8c1 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:18.337720 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d55ed-0c5a-4220-8069-183faa33f8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:18.337966 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d55ed-0c5a-4220-8069-183faa33f8c1 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:18.339654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 24/42] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:22:18.363672 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f4951-ab84-4ebc-9102-621fdfd223b6 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:18.363998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f4951-ab84-4ebc-9102-621fdfd223b6 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:18.364257 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f4951-ab84-4ebc-9102-621fdfd223b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:18.388600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-7d6f4951-ab84-4ebc-9102-621fdfd223b6 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:18.389774 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7d6f4951-ab84-4ebc-9102-621fdfd223b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:18.441713 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774622b3-f663-4505-b28a-b3a4996ef229 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:18.480697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 25/43] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:22:18.483102 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8adab5-887e-44df-a43a-d0ad6a5a92aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:18.483792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8adab5-887e-44df-a43a-d0ad6a5a92aa None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:18.484088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8adab5-887e-44df-a43a-d0ad6a5a92aa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:18.484382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8adab5-887e-44df-a43a-d0ad6a5a92aa None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:18.485217 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8adab5-887e-44df-a43a-d0ad6a5a92aa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:18.486618 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8adab5-887e-44df-a43a-d0ad6a5a92aa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:18.502040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 19/44] 10.222.0.43 () {60 vars in 1131 bytes} [Thu Jun 23 22:22:18 2022] POST /identity/v3/roles => generated 223 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:22:18.505284 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f4951-ab84-4ebc-9102-621fdfd223b6 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:18.550647 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:18.637937 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f4951-ab84-4ebc-9102-621fdfd223b6 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:18.786796 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f960e4-23b6-4100-8a47-d8254226a8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:18.787541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f960e4-23b6-4100-8a47-d8254226a8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:18.787832 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f960e4-23b6-4100-8a47-d8254226a8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:18.788116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f960e4-23b6-4100-8a47-d8254226a8b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:18.790281 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f960e4-23b6-4100-8a47-d8254226a8b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:18.791754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f960e4-23b6-4100-8a47-d8254226a8b5 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:18.793368 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 26/45] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:22:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:18.799409 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8adab5-887e-44df-a43a-d0ad6a5a92aa None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:18.868307 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:18.903461 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8adab5-887e-44df-a43a-d0ad6a5a92aa None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:18.936345 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73f36d-9efb-4893-b78f-b8bef56ed871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:18.937016 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73f36d-9efb-4893-b78f-b8bef56ed871 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:18.937303 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73f36d-9efb-4893-b78f-b8bef56ed871 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:18.937584 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73f36d-9efb-4893-b78f-b8bef56ed871 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:18.938305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73f36d-9efb-4893-b78f-b8bef56ed871 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:18.939802 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73f36d-9efb-4893-b78f-b8bef56ed871 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:18.967668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 20/46] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 22:22:18 2022] POST /identity/v3/projects => generated 312 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:22:20.226053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b0491-0323-4f7d-a209-06aa6aa095c5 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:20.227110 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b0491-0323-4f7d-a209-06aa6aa095c5 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:20.227188 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b0491-0323-4f7d-a209-06aa6aa095c5 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:20.228612 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 27/47] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:20.233927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532e1c7-ada7-468b-9342-bc279b8336e7 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:20.234328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532e1c7-ada7-468b-9342-bc279b8336e7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:20.234503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532e1c7-ada7-468b-9342-bc279b8336e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:20.354348 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a532e1c7-ada7-468b-9342-bc279b8336e7 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:20.355896 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a532e1c7-ada7-468b-9342-bc279b8336e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:20.421823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 21/48] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:20.449852 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b0491-0323-4f7d-a209-06aa6aa095c5 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:20.549755 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501b0491-0323-4f7d-a209-06aa6aa095c5 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:20.552073 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:20.583954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3a74c-8631-4f30-bfbc-c54beaf3661c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:20.584758 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3a74c-8631-4f30-bfbc-c54beaf3661c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:20.585045 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3a74c-8631-4f30-bfbc-c54beaf3661c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:20.585327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3a74c-8631-4f30-bfbc-c54beaf3661c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:20.597281 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3a74c-8631-4f30-bfbc-c54beaf3661c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:20.605202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-19a3a74c-8631-4f30-bfbc-c54beaf3661c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:22:20.605202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:20.605202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:20.605202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:20.605202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:20.605202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:20.605202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:20.605202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:20.605202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:20.605202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:20.605202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:20.605202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:20.605202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:20.605202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:20.605202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:20.605202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:20.606091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:20.606091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:20.606091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:20.606091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:20.606091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:20.606091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:20.606091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:20.606091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:20.606091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:20.606091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:20.606091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:20.606091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:20.606091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:20.606091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:20.606091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:20.606091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:20.606840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:20.606840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:20.606840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:20.606840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:20.606840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:20.606840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:20.606840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:20.606840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:20.606840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:20.606840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:20.606840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:22:20.606840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:20.606840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 28/49] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:22:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:20.612209 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532e1c7-ada7-468b-9342-bc279b8336e7 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:20.660485 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:20.712627 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532e1c7-ada7-468b-9342-bc279b8336e7 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:20.745827 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767f2f8-758e-490c-9d2b-c4b99134030d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:20.746494 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767f2f8-758e-490c-9d2b-c4b99134030d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:20.746785 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767f2f8-758e-490c-9d2b-c4b99134030d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:20.747112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767f2f8-758e-490c-9d2b-c4b99134030d None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:20.747860 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767f2f8-758e-490c-9d2b-c4b99134030d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:20.748172 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767f2f8-758e-490c-9d2b-c4b99134030d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:20.749555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767f2f8-758e-490c-9d2b-c4b99134030d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:20.760940 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 22/50] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:20.766379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3a74c-8631-4f30-bfbc-c54beaf3661c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:20.865240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3a74c-8631-4f30-bfbc-c54beaf3661c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:20.898806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735f5a3-c887-4898-8fe7-09c590aa995b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:20.899721 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735f5a3-c887-4898-8fe7-09c590aa995b None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:20.900018 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735f5a3-c887-4898-8fe7-09c590aa995b None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:20.900305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735f5a3-c887-4898-8fe7-09c590aa995b None admin] PATH_INFO: `/v3/users/c3c5fce3a5074734811c3407aac7fd6b` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:20.901500 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735f5a3-c887-4898-8fe7-09c590aa995b None admin] RBAC: Authorizing `identity:get_user(user_id=c3c5fce3a5074734811c3407aac7fd6b)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:20.902971 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735f5a3-c887-4898-8fe7-09c590aa995b None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:20.904605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 29/51] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:20 2022] GET /identity/v3/users/c3c5fce3a5074734811c3407aac7fd6b => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:20.909961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767f2f8-758e-490c-9d2b-c4b99134030d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:21.008560 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767f2f8-758e-490c-9d2b-c4b99134030d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:21.018879 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:21.029358 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fae04-689e-46cd-b61a-36cbecdec007 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:21.029651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fae04-689e-46cd-b61a-36cbecdec007 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:21.029945 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fae04-689e-46cd-b61a-36cbecdec007 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:21.031498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 30/52] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:22:21.036171 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2e126-ee07-4159-ac78-f61d005025c3 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:21.036467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2e126-ee07-4159-ac78-f61d005025c3 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:21.036755 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2e126-ee07-4159-ac78-f61d005025c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:21.045981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559725d-60d3-4bb7-8619-84b854597a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:21.046784 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559725d-60d3-4bb7-8619-84b854597a07 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:21.047100 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559725d-60d3-4bb7-8619-84b854597a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:21.047382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559725d-60d3-4bb7-8619-84b854597a07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:21.048565 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559725d-60d3-4bb7-8619-84b854597a07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:21.050034 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559725d-60d3-4bb7-8619-84b854597a07 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:21.051696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 23/53] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:22:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:21.053217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-0fb2e126-ee07-4159-ac78-f61d005025c3 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:21.055017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-0fb2e126-ee07-4159-ac78-f61d005025c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:21.057686 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559725d-60d3-4bb7-8619-84b854597a07 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:21.082456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:21.123795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 31/54] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:22:21.147197 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2e126-ee07-4159-ac78-f61d005025c3 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:21.160345 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559725d-60d3-4bb7-8619-84b854597a07 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:21.194205 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d915e-9346-450d-bb2d-8d7e37860420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:21.195431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d915e-9346-450d-bb2d-8d7e37860420 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:21.195844 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d915e-9346-450d-bb2d-8d7e37860420 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:21.196149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d915e-9346-450d-bb2d-8d7e37860420 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:21.197011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d915e-9346-450d-bb2d-8d7e37860420 None admin] RBAC: Adding query filter params (role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.domain.id=default, user.id=c3c5fce3a5074734811c3407aac7fd6b) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:21.197340 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d915e-9346-450d-bb2d-8d7e37860420 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:21.198790 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d915e-9346-450d-bb2d-8d7e37860420 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:21.208038 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 24/55] 10.222.0.43 () {56 vars in 1322 bytes} [Thu Jun 23 22:22:21 2022] GET /identity/v3/role_assignments?user.id=c3c5fce3a5074734811c3407aac7fd6b&scope.domain.id=default&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:21.247799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2e126-ee07-4159-ac78-f61d005025c3 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:21.284499 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e9c17-2c26-4944-8624-eeaa96ab5fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:21.285242 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e9c17-2c26-4944-8624-eeaa96ab5fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:21.285536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e9c17-2c26-4944-8624-eeaa96ab5fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:21.285824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e9c17-2c26-4944-8624-eeaa96ab5fb6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:21.287104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e9c17-2c26-4944-8624-eeaa96ab5fb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:21.288754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e9c17-2c26-4944-8624-eeaa96ab5fb6 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:21.290915 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 32/56] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:22:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:21.296589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d915e-9346-450d-bb2d-8d7e37860420 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:21.342612 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:21.403850 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d915e-9346-450d-bb2d-8d7e37860420 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:21.438041 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0718ab-e284-4ec7-9a57-c446ebd7aac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:21.438716 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0718ab-e284-4ec7-9a57-c446ebd7aac1 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:21.439041 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0718ab-e284-4ec7-9a57-c446ebd7aac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:21.439334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0718ab-e284-4ec7-9a57-c446ebd7aac1 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:21.440054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0718ab-e284-4ec7-9a57-c446ebd7aac1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:21.441522 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0718ab-e284-4ec7-9a57-c446ebd7aac1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:21.467107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 25/57] 10.222.0.43 () {60 vars in 1140 bytes} [Thu Jun 23 22:22:21 2022] POST /identity/v3/projects => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:22:23.511609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88a746-6740-4254-ab25-fd0d7be05ac6 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:23.512124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88a746-6740-4254-ab25-fd0d7be05ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:23.512230 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88a746-6740-4254-ab25-fd0d7be05ac6 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:23.513566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 33/58] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:23.518667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b0eac-602e-4bf9-8c83-90baf33ab1ec None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:23.519081 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b0eac-602e-4bf9-8c83-90baf33ab1ec None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:23.519273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b0eac-602e-4bf9-8c83-90baf33ab1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:23.535945 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9a1b0eac-602e-4bf9-8c83-90baf33ab1ec None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:23.537150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9a1b0eac-602e-4bf9-8c83-90baf33ab1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:23.602423 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 26/59] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:23.626282 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88a746-6740-4254-ab25-fd0d7be05ac6 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:23.656472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:23.727226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88a746-6740-4254-ab25-fd0d7be05ac6 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:23.761097 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9965d8-b972-446c-94a5-1a98fa553d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:23.761941 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9965d8-b972-446c-94a5-1a98fa553d72 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:23.762183 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9965d8-b972-446c-94a5-1a98fa553d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:23.762466 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9965d8-b972-446c-94a5-1a98fa553d72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:23.763668 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9965d8-b972-446c-94a5-1a98fa553d72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:23.765128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9965d8-b972-446c-94a5-1a98fa553d72 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:23.766734 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 34/60] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:22:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:23.772466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b0eac-602e-4bf9-8c83-90baf33ab1ec None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:23.797151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:23.871040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b0eac-602e-4bf9-8c83-90baf33ab1ec None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:23.903127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76229ba-5731-4a64-b07f-81259dce15a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:23.903800 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76229ba-5731-4a64-b07f-81259dce15a5 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:23.904094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76229ba-5731-4a64-b07f-81259dce15a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:23.904382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76229ba-5731-4a64-b07f-81259dce15a5 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:23.905093 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76229ba-5731-4a64-b07f-81259dce15a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:23.906577 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76229ba-5731-4a64-b07f-81259dce15a5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:23.932070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 27/61] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 22:22:23 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:22:27.011208 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024ddc3-f895-4042-bd86-1a77fad93429 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:27.012243 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024ddc3-f895-4042-bd86-1a77fad93429 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:27.012742 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024ddc3-f895-4042-bd86-1a77fad93429 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:27.014324 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 35/62] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:27.019581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e1fbd-a6d1-4300-a9c7-bce8f1503252 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:27.020392 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e1fbd-a6d1-4300-a9c7-bce8f1503252 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:27.020899 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e1fbd-a6d1-4300-a9c7-bce8f1503252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:27.047920 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-eb7e1fbd-a6d1-4300-a9c7-bce8f1503252 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:27.049477 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-eb7e1fbd-a6d1-4300-a9c7-bce8f1503252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:27.117471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b8149-1cd9-4860-a90a-9dfa244f03fe None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:27.118092 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b8149-1cd9-4860-a90a-9dfa244f03fe None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:27.118593 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b8149-1cd9-4860-a90a-9dfa244f03fe None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:27.120193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 36/63] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:22:27.132196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7ccb3-1da1-48c8-a99e-063a07354192 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:27.133345 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7ccb3-1da1-48c8-a99e-063a07354192 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:27.133848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7ccb3-1da1-48c8-a99e-063a07354192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:27.162332 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-2ba7ccb3-1da1-48c8-a99e-063a07354192 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:27.165142 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2ba7ccb3-1da1-48c8-a99e-063a07354192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:27.173386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 28/64] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:27.199652 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e1fbd-a6d1-4300-a9c7-bce8f1503252 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:27.222360 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:27.306550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e1fbd-a6d1-4300-a9c7-bce8f1503252 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:27.322506 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 37/65] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:27.324739 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098c7fd-eddd-4113-a79e-afbf1d457a83 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:27.325189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098c7fd-eddd-4113-a79e-afbf1d457a83 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:27.325873 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098c7fd-eddd-4113-a79e-afbf1d457a83 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:27.327819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 38/66] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:27.333036 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad075732-4cfb-4ce0-aaab-0c5ca6641a91 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:27.333330 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad075732-4cfb-4ce0-aaab-0c5ca6641a91 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:27.333624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad075732-4cfb-4ce0-aaab-0c5ca6641a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:27.360737 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-ad075732-4cfb-4ce0-aaab-0c5ca6641a91 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:27.361061 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60512e9f-5ddf-4dd4-9611-0885879d3845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:27.362049 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60512e9f-5ddf-4dd4-9611-0885879d3845 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:27.362486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-ad075732-4cfb-4ce0-aaab-0c5ca6641a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:27.362618 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60512e9f-5ddf-4dd4-9611-0885879d3845 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:27.363064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60512e9f-5ddf-4dd4-9611-0885879d3845 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:27.378472 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60512e9f-5ddf-4dd4-9611-0885879d3845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:27.390556 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:27.391834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-60512e9f-5ddf-4dd4-9611-0885879d3845 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:27.391834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:27.391834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:27.391834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:27.391834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:27.391834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:27.391834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:27.391834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:27.391834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:27.391834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:27.391834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:27.391834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:27.391834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:27.391834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:27.391834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:27.391834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:27.392926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:27.392926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:27.392926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:27.392926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:27.392926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:27.392926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:27.392926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:27.392926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:27.392926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:27.392926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:27.392926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:27.392926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:27.392926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:27.392926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:27.392926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:27.392926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:27.393813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:27.393813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:27.393813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:27.393813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:27.393813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:27.393813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:27.393813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:27.393813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:27.393813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:27.393813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:27.393813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:27.393813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:22:27.396579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 29/67] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:22:27.398208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60512e9f-5ddf-4dd4-9611-0885879d3845 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:27.450073 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 39/68] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:22:27.451744 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad075732-4cfb-4ce0-aaab-0c5ca6641a91 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:27.503283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:27.535096 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60512e9f-5ddf-4dd4-9611-0885879d3845 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:27.573471 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b72cb-0551-45ef-93d8-e48c63ca04fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:27.574190 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b72cb-0551-45ef-93d8-e48c63ca04fc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:27.574478 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b72cb-0551-45ef-93d8-e48c63ca04fc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:27.574759 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b72cb-0551-45ef-93d8-e48c63ca04fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:27.585417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b72cb-0551-45ef-93d8-e48c63ca04fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:27.585590 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad075732-4cfb-4ce0-aaab-0c5ca6641a91 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:27.593531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-086b72cb-0551-45ef-93d8-e48c63ca04fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:22:27.593531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:27.593531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:27.593531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:27.593531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:27.593531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:27.593531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:27.593531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:27.593531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:27.593531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:27.593531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:27.593531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:27.593531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:27.593531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:27.593531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:27.593531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:27.594591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:27.594591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:27.594591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:27.594591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:27.594591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:27.594591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:27.594591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:27.594591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:27.594591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:27.594591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:27.594591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:27.594591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:27.594591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:27.594591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:27.594591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:27.594591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:27.595566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:27.595566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:27.595566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:27.595566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:27.595566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:27.595566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:27.595566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:27.595566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:27.595566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:27.595566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:27.595566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:22:27.595566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:22:27.595566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 30/69] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:22:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:27.595566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b72cb-0551-45ef-93d8-e48c63ca04fc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:27.626744 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bffd0-662a-4357-aa02-9116390f64e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:27.627694 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bffd0-662a-4357-aa02-9116390f64e7 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:27.628144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bffd0-662a-4357-aa02-9116390f64e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:27.628571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bffd0-662a-4357-aa02-9116390f64e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:27.629468 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bffd0-662a-4357-aa02-9116390f64e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:27.629926 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bffd0-662a-4357-aa02-9116390f64e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:27.631489 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bffd0-662a-4357-aa02-9116390f64e7 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:27.640502 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 40/70] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:27.641907 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94151f-6ae9-4c6b-bea8-dfe608965ef0 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:27.642349 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94151f-6ae9-4c6b-bea8-dfe608965ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:27.642777 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94151f-6ae9-4c6b-bea8-dfe608965ef0 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:27.644189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 41/71] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:27.645021 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94151f-6ae9-4c6b-bea8-dfe608965ef0 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:27.670327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:27.708993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b72cb-0551-45ef-93d8-e48c63ca04fc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:27.738140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:27.748412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff058048-8e3f-45a7-b592-6dbeff86298f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:27.749438 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff058048-8e3f-45a7-b592-6dbeff86298f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:27.749731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff058048-8e3f-45a7-b592-6dbeff86298f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:27.750004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff058048-8e3f-45a7-b592-6dbeff86298f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:27.760591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff058048-8e3f-45a7-b592-6dbeff86298f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:27.768695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ff058048-8e3f-45a7-b592-6dbeff86298f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:22:27.768695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:27.768695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:27.768695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:27.768695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:27.768695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:27.768695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:27.768695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:27.768695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:27.768695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:27.768695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:27.768695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:27.768695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:27.768695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:27.768695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:27.768695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:27.769571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:27.769571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:27.769571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:27.769571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:27.769571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:27.769571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:27.769571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:27.769571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:27.769571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:27.769571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:27.769571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:27.769571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:27.769571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:27.769571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:27.769571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:27.769571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:27.770311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:27.770311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:27.770311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:27.770311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:27.770311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:27.770311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:27.770311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:27.770311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:27.770311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:27.770311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:27.770311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:22:27.770311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:22:27.770887 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94151f-6ae9-4c6b-bea8-dfe608965ef0 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:27.773322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 31/72] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:22:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:27.773322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff058048-8e3f-45a7-b592-6dbeff86298f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:27.817781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45992e-a0d5-445e-a722-e913ac6c84b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:27.818510 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45992e-a0d5-445e-a722-e913ac6c84b5 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:27.818803 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45992e-a0d5-445e-a722-e913ac6c84b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:27.819768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45992e-a0d5-445e-a722-e913ac6c84b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:27.820188 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c45992e-a0d5-445e-a722-e913ac6c84b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:27.820584 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c45992e-a0d5-445e-a722-e913ac6c84b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:27.822199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c45992e-a0d5-445e-a722-e913ac6c84b5 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:27.837418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 42/73] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:27.837418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b141a-7349-472a-b3e7-db84132a36ab None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:27.837418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b141a-7349-472a-b3e7-db84132a36ab None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:27.837418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b141a-7349-472a-b3e7-db84132a36ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:27.852841 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-491b141a-7349-472a-b3e7-db84132a36ab None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:27.854391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-491b141a-7349-472a-b3e7-db84132a36ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:27.888567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff058048-8e3f-45a7-b592-6dbeff86298f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:27.924504 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb1677-79d8-4af5-83b0-d95cf66fd1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:27.925390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb1677-79d8-4af5-83b0-d95cf66fd1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:27.925689 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb1677-79d8-4af5-83b0-d95cf66fd1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:27.925976 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb1677-79d8-4af5-83b0-d95cf66fd1d8 None admin] PATH_INFO: `/v3/users/ae81942c6123431c8ac728c83a3710b6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:27.927886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 43/74] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:27.928769 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b141a-7349-472a-b3e7-db84132a36ab None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:27.952121 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:27.952819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb1677-79d8-4af5-83b0-d95cf66fd1d8 None admin] RBAC: Authorizing `identity:get_user(user_id=ae81942c6123431c8ac728c83a3710b6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:27.954479 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acb1677-79d8-4af5-83b0-d95cf66fd1d8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:27.961119 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 32/75] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:27 2022] GET /identity/v3/users/ae81942c6123431c8ac728c83a3710b6 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:27.961912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb1677-79d8-4af5-83b0-d95cf66fd1d8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:28.011754 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:28.032435 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b141a-7349-472a-b3e7-db84132a36ab None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:28.064363 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acb1677-79d8-4af5-83b0-d95cf66fd1d8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:28.067080 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f68f2c-4654-47ee-b54a-8a1e15aca3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:28.067763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f68f2c-4654-47ee-b54a-8a1e15aca3cd None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:28.068056 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f68f2c-4654-47ee-b54a-8a1e15aca3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:28.068346 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f68f2c-4654-47ee-b54a-8a1e15aca3cd None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:28.069106 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f68f2c-4654-47ee-b54a-8a1e15aca3cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:28.069424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f68f2c-4654-47ee-b54a-8a1e15aca3cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:28.071064 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f68f2c-4654-47ee-b54a-8a1e15aca3cd None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:28.078897 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 44/76] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:22:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:28.080464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766e98c-c633-43c9-b6d0-173952cb79b6 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:28.080760 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766e98c-c633-43c9-b6d0-173952cb79b6 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:28.081048 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766e98c-c633-43c9-b6d0-173952cb79b6 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:28.082303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 45/77] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:28.083082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766e98c-c633-43c9-b6d0-173952cb79b6 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:28.103010 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41533f-3ae8-4b3a-8012-735e21d3d9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:28.103902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41533f-3ae8-4b3a-8012-735e21d3d9f8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:28.104270 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41533f-3ae8-4b3a-8012-735e21d3d9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:28.104620 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41533f-3ae8-4b3a-8012-735e21d3d9f8 None admin] PATH_INFO: `/v3/users/c3c5fce3a5074734811c3407aac7fd6b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:28.105889 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41533f-3ae8-4b3a-8012-735e21d3d9f8 None admin] RBAC: Authorizing `identity:get_user(user_id=c3c5fce3a5074734811c3407aac7fd6b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:28.107381 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41533f-3ae8-4b3a-8012-735e21d3d9f8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:28.108991 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 33/78] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:27 2022] GET /identity/v3/users/c3c5fce3a5074734811c3407aac7fd6b => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:28.109768 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41533f-3ae8-4b3a-8012-735e21d3d9f8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:28.168584 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:28.193964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766e98c-c633-43c9-b6d0-173952cb79b6 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:28.212104 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41533f-3ae8-4b3a-8012-735e21d3d9f8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:28.228103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:28.228454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbff394-d751-42d1-ab32-975d0c3ee520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:28.229369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbff394-d751-42d1-ab32-975d0c3ee520 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:28.229564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbff394-d751-42d1-ab32-975d0c3ee520 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:28.229874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbff394-d751-42d1-ab32-975d0c3ee520 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:28.240409 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbff394-d751-42d1-ab32-975d0c3ee520 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:28.246894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11324057-a356-4188-bae3-3e2afac51b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:28.247574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11324057-a356-4188-bae3-3e2afac51b3d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:28.247869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11324057-a356-4188-bae3-3e2afac51b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:28.248156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11324057-a356-4188-bae3-3e2afac51b3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:28.248770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-abbff394-d751-42d1-ab32-975d0c3ee520 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:28.248770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:28.248770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:28.248770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:28.248770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:28.248770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:28.248770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:28.248770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:28.248770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:28.248770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:28.248770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:28.248770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:28.248770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:28.248770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:28.248770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:28.248770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:28.249575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:28.249575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:28.249575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:28.249575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:28.249575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:28.249575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:28.249575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:28.249575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:28.249575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:28.249575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:28.249575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:28.249575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:28.249575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:28.249575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:28.249575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:28.249575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:28.250350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:28.250350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:28.250350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:28.250350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:28.250350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:28.250350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:28.250350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:28.250350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:28.250350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:28.250350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:28.250350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:28.250350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:28.250350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 46/79] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:28.251007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbff394-d751-42d1-ab32-975d0c3ee520 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:28.316656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11324057-a356-4188-bae3-3e2afac51b3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:28.318121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11324057-a356-4188-bae3-3e2afac51b3d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:28.349471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbff394-d751-42d1-ab32-975d0c3ee520 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:28.382046 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55337539-33e8-4879-bcda-bf54dab280f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:28.388948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55337539-33e8-4879-bcda-bf54dab280f4 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:28.389258 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55337539-33e8-4879-bcda-bf54dab280f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:28.389551 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55337539-33e8-4879-bcda-bf54dab280f4 None admin] PATH_INFO: `/v3/users/ae81942c6123431c8ac728c83a3710b6` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:28.390842 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55337539-33e8-4879-bcda-bf54dab280f4 None admin] RBAC: Authorizing `identity:get_user(user_id=ae81942c6123431c8ac728c83a3710b6)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:28.392320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55337539-33e8-4879-bcda-bf54dab280f4 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:28.393995 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 47/80] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:28 2022] GET /identity/v3/users/ae81942c6123431c8ac728c83a3710b6 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:28.395186 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13605493-7c03-404b-ac6d-5ff95c51d465 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:28.395480 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13605493-7c03-404b-ac6d-5ff95c51d465 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:28.395769 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13605493-7c03-404b-ac6d-5ff95c51d465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:28.401880 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:28.414950 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-13605493-7c03-404b-ac6d-5ff95c51d465 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:28.416205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-13605493-7c03-404b-ac6d-5ff95c51d465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:28.425807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 34/81] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:22:28.426654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11324057-a356-4188-bae3-3e2afac51b3d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:28.444425 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:28.484613 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 48/82] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:28.485457 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13605493-7c03-404b-ac6d-5ff95c51d465 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:28.528129 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11324057-a356-4188-bae3-3e2afac51b3d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:28.562724 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2942646-9dcd-4daf-896e-a019c5df0280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:28.563405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2942646-9dcd-4daf-896e-a019c5df0280 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:28.563699 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2942646-9dcd-4daf-896e-a019c5df0280 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:28.563987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2942646-9dcd-4daf-896e-a019c5df0280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:28.589654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13605493-7c03-404b-ac6d-5ff95c51d465 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:28.619609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:28.625790 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad730b2-d0fa-405d-bc48-4e14f2516e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:28.626473 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad730b2-d0fa-405d-bc48-4e14f2516e02 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:28.626765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad730b2-d0fa-405d-bc48-4e14f2516e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:28.627086 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad730b2-d0fa-405d-bc48-4e14f2516e02 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:28.627840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad730b2-d0fa-405d-bc48-4e14f2516e02 None admin] RBAC: Adding query filter params (name=member) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:28.628163 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad730b2-d0fa-405d-bc48-4e14f2516e02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:28.629543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad730b2-d0fa-405d-bc48-4e14f2516e02 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:28.631426 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2942646-9dcd-4daf-896e-a019c5df0280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:28.633155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2942646-9dcd-4daf-896e-a019c5df0280 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:28.638076 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 49/83] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:28.638871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad730b2-d0fa-405d-bc48-4e14f2516e02 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:28.651615 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:28.736371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 35/84] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:22:28.737436 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2942646-9dcd-4daf-896e-a019c5df0280 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:28.740819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad730b2-d0fa-405d-bc48-4e14f2516e02 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:28.774774 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d0084-f46e-4d18-beb9-ad9387c1399e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:28.775471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d0084-f46e-4d18-beb9-ad9387c1399e None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:28.775752 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d0084-f46e-4d18-beb9-ad9387c1399e None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:28.776038 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d0084-f46e-4d18-beb9-ad9387c1399e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:28.834724 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:28.837409 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2942646-9dcd-4daf-896e-a019c5df0280 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:28.842650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d0084-f46e-4d18-beb9-ad9387c1399e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:28.844179 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d0084-f46e-4d18-beb9-ad9387c1399e None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:28.855223 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:28.873710 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36b534-954a-46cd-9665-a8c282c107dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:28.874572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36b534-954a-46cd-9665-a8c282c107dc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:28.874869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36b534-954a-46cd-9665-a8c282c107dc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:28.875185 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36b534-954a-46cd-9665-a8c282c107dc None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:28.876381 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36b534-954a-46cd-9665-a8c282c107dc None admin] RBAC: Authorizing `identity:get_project(project_id=53390afb4a404df79ab2d9ae31e8d058)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:28.877838 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36b534-954a-46cd-9665-a8c282c107dc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:28.879577 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 36/85] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:28 2022] GET /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:28.880352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36b534-954a-46cd-9665-a8c282c107dc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:28.945230 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 50/86] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:22:28.946538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d0084-f46e-4d18-beb9-ad9387c1399e None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:28.982814 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36b534-954a-46cd-9665-a8c282c107dc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:29.016049 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237ddf9-612d-4ea2-bfd8-8d07bc93519f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:29.016837 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237ddf9-612d-4ea2-bfd8-8d07bc93519f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:29.017129 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237ddf9-612d-4ea2-bfd8-8d07bc93519f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:29.017409 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237ddf9-612d-4ea2-bfd8-8d07bc93519f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:29.018582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237ddf9-612d-4ea2-bfd8-8d07bc93519f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:29.020054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4237ddf9-612d-4ea2-bfd8-8d07bc93519f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:29.021903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 37/87] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:22:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:29.026937 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237ddf9-612d-4ea2-bfd8-8d07bc93519f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:29.042906 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:29.050360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d0084-f46e-4d18-beb9-ad9387c1399e None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:29.074609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:29.085023 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f34c35-478a-424e-8584-6b6f891a45e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:29.085965 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f34c35-478a-424e-8584-6b6f891a45e4 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:29.086258 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f34c35-478a-424e-8584-6b6f891a45e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:29.086550 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f34c35-478a-424e-8584-6b6f891a45e4 None admin] PATH_INFO: `/v3/users/ae81942c6123431c8ac728c83a3710b6` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:29.087811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f34c35-478a-424e-8584-6b6f891a45e4 None admin] RBAC: Authorizing `identity:get_user(user_id=ae81942c6123431c8ac728c83a3710b6)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:29.089281 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f34c35-478a-424e-8584-6b6f891a45e4 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:29.090947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 51/88] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:28 2022] GET /identity/v3/users/ae81942c6123431c8ac728c83a3710b6 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:29.091713 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f34c35-478a-424e-8584-6b6f891a45e4 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:29.122428 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237ddf9-612d-4ea2-bfd8-8d07bc93519f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:29.159075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab205b8-c4b5-4e9f-9d9b-1440a79fcdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:29.159951 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab205b8-c4b5-4e9f-9d9b-1440a79fcdef None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:29.160262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab205b8-c4b5-4e9f-9d9b-1440a79fcdef None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:29.160533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab205b8-c4b5-4e9f-9d9b-1440a79fcdef None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:29.161742 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab205b8-c4b5-4e9f-9d9b-1440a79fcdef None admin] RBAC: Authorizing `identity:get_project(project_id=53390afb4a404df79ab2d9ae31e8d058)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:29.163232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab205b8-c4b5-4e9f-9d9b-1440a79fcdef None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:29.165681 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 38/89] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:29 2022] GET /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:29.166458 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab205b8-c4b5-4e9f-9d9b-1440a79fcdef None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:29.194120 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f34c35-478a-424e-8584-6b6f891a45e4 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:29.227954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d44735-9c16-4a46-bb2c-1d2b01a83b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:29.228616 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d44735-9c16-4a46-bb2c-1d2b01a83b85 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:29.228904 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d44735-9c16-4a46-bb2c-1d2b01a83b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:29.229194 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d44735-9c16-4a46-bb2c-1d2b01a83b85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:29.230024 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d44735-9c16-4a46-bb2c-1d2b01a83b85 None admin] RBAC: Adding query filter params (role.id=1baba2239751440eb25af0d0fd2d072d, scope.project.id=53390afb4a404df79ab2d9ae31e8d058, user.id=ae81942c6123431c8ac728c83a3710b6) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:29.230343 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d44735-9c16-4a46-bb2c-1d2b01a83b85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:29.231810 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d44735-9c16-4a46-bb2c-1d2b01a83b85 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:29.237086 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 52/90] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:22:29 2022] GET /identity/v3/role_assignments?user.id=ae81942c6123431c8ac728c83a3710b6&scope.project.id=53390afb4a404df79ab2d9ae31e8d058&role.id=1baba2239751440eb25af0d0fd2d072d => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:29.237840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d44735-9c16-4a46-bb2c-1d2b01a83b85 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:29.268688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:29.277645 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab205b8-c4b5-4e9f-9d9b-1440a79fcdef None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:29.302541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:29.321052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3acde-a9bb-45e1-b9a7-11b222f22c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:29.321952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3acde-a9bb-45e1-b9a7-11b222f22c9c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:29.322243 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3acde-a9bb-45e1-b9a7-11b222f22c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:29.322529 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec3acde-a9bb-45e1-b9a7-11b222f22c9c None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:29.323878 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3acde-a9bb-45e1-b9a7-11b222f22c9c None admin] RBAC: Authorizing `identity:get_project(project_id=53390afb4a404df79ab2d9ae31e8d058)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:29.325382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec3acde-a9bb-45e1-b9a7-11b222f22c9c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:29.332094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 39/91] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:29 2022] GET /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:29.332094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3acde-a9bb-45e1-b9a7-11b222f22c9c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:29.364615 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d44735-9c16-4a46-bb2c-1d2b01a83b85 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:29.407022 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e4bbd-77cf-49d3-a795-4b7262dac061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:29.407949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e4bbd-77cf-49d3-a795-4b7262dac061 None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:29.408380 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e4bbd-77cf-49d3-a795-4b7262dac061 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:29.408804 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e4bbd-77cf-49d3-a795-4b7262dac061 None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:29.409696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e4bbd-77cf-49d3-a795-4b7262dac061 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:29.411376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e4bbd-77cf-49d3-a795-4b7262dac061 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:29.442961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 53/92] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 22:22:29 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:22:29.444019 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e4bbd-77cf-49d3-a795-4b7262dac061 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:29.452797 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec3acde-a9bb-45e1-b9a7-11b222f22c9c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:29.497136 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043cbfa-e39f-4b02-85a7-299b2cdc7506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:29.498050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043cbfa-e39f-4b02-85a7-299b2cdc7506 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:29.498484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043cbfa-e39f-4b02-85a7-299b2cdc7506 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:29.498907 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043cbfa-e39f-4b02-85a7-299b2cdc7506 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:29.551325 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:29.572086 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:29.579291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e4bbd-77cf-49d3-a795-4b7262dac061 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:29.588089 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043cbfa-e39f-4b02-85a7-299b2cdc7506 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:29.589981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043cbfa-e39f-4b02-85a7-299b2cdc7506 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:29.618121 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4c477-0adc-483c-9449-60f7ed7751d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:29.618831 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4c477-0adc-483c-9449-60f7ed7751d2 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:29.619147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4c477-0adc-483c-9449-60f7ed7751d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:29.619512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4c477-0adc-483c-9449-60f7ed7751d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:29.620370 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4c477-0adc-483c-9449-60f7ed7751d2 None admin] RBAC: Adding query filter params (role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.project.id=53390afb4a404df79ab2d9ae31e8d058, user.id=c3c5fce3a5074734811c3407aac7fd6b) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:29.620694 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4c477-0adc-483c-9449-60f7ed7751d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:29.622238 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4c477-0adc-483c-9449-60f7ed7751d2 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:29.629255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 54/93] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:22:29 2022] GET /identity/v3/role_assignments?user.id=c3c5fce3a5074734811c3407aac7fd6b&scope.project.id=53390afb4a404df79ab2d9ae31e8d058&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:29.630158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4c477-0adc-483c-9449-60f7ed7751d2 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:29.731057 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 40/94] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:22:29.736952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043cbfa-e39f-4b02-85a7-299b2cdc7506 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:29.812152 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4c477-0adc-483c-9449-60f7ed7751d2 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:29.862713 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e510d1c-2ea8-44a1-a80a-2f608e53245b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:29.863423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e510d1c-2ea8-44a1-a80a-2f608e53245b None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:29.863716 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e510d1c-2ea8-44a1-a80a-2f608e53245b None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:29.864120 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e510d1c-2ea8-44a1-a80a-2f608e53245b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:29.865073 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e510d1c-2ea8-44a1-a80a-2f608e53245b None admin] RBAC: Adding query filter params (role.id=61397d4a1ccd4a8ca479f98186ed489b, scope.project.id=53390afb4a404df79ab2d9ae31e8d058, user.id=ae81942c6123431c8ac728c83a3710b6) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:29.865398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e510d1c-2ea8-44a1-a80a-2f608e53245b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:29.867722 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e510d1c-2ea8-44a1-a80a-2f608e53245b None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:29.876484 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 55/95] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:22:29 2022] GET /identity/v3/role_assignments?user.id=ae81942c6123431c8ac728c83a3710b6&scope.project.id=53390afb4a404df79ab2d9ae31e8d058&role.id=61397d4a1ccd4a8ca479f98186ed489b => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:29.879192 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:29.894155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043cbfa-e39f-4b02-85a7-299b2cdc7506 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:29.937393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55255117-c3ec-49eb-a0de-2db834eb8795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:29.938270 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55255117-c3ec-49eb-a0de-2db834eb8795 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:29.938560 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55255117-c3ec-49eb-a0de-2db834eb8795 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:29.938845 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55255117-c3ec-49eb-a0de-2db834eb8795 None admin] PATH_INFO: `/v3/projects/ff7c7330c1444db4a0c394169b7517d0` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:29.940159 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55255117-c3ec-49eb-a0de-2db834eb8795 None admin] RBAC: Authorizing `identity:get_project(project_id=ff7c7330c1444db4a0c394169b7517d0)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:29.941656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55255117-c3ec-49eb-a0de-2db834eb8795 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:29.943512 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 41/96] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:29 2022] GET /identity/v3/projects/ff7c7330c1444db4a0c394169b7517d0 => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:29.950401 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e510d1c-2ea8-44a1-a80a-2f608e53245b None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:29.965537 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:30.073900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e510d1c-2ea8-44a1-a80a-2f608e53245b None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:30.113926 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508dbfd6-9b67-4b3c-baef-79d2db1e7db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:30.114605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dbfd6-9b67-4b3c-baef-79d2db1e7db4 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:30.114903 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dbfd6-9b67-4b3c-baef-79d2db1e7db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:30.115221 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dbfd6-9b67-4b3c-baef-79d2db1e7db4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:30.116093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508dbfd6-9b67-4b3c-baef-79d2db1e7db4 None admin] RBAC: Adding query filter params (role.id=1baba2239751440eb25af0d0fd2d072d, scope.project.id=ff7c7330c1444db4a0c394169b7517d0, user.id=ae81942c6123431c8ac728c83a3710b6) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:30.116393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508dbfd6-9b67-4b3c-baef-79d2db1e7db4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:30.118703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508dbfd6-9b67-4b3c-baef-79d2db1e7db4 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:30.125342 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 56/97] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:22:29 2022] GET /identity/v3/role_assignments?user.id=ae81942c6123431c8ac728c83a3710b6&scope.project.id=ff7c7330c1444db4a0c394169b7517d0&role.id=1baba2239751440eb25af0d0fd2d072d => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:31.776553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb9d59-2eb9-4c43-b86d-313552afd010 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:31.777059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb9d59-2eb9-4c43-b86d-313552afd010 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:31.777132 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb9d59-2eb9-4c43-b86d-313552afd010 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:31.778511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 42/98] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:31.788151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc2004-8a17-4d54-8ccb-992b4341f790 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:31.788571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc2004-8a17-4d54-8ccb-992b4341f790 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:31.788767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc2004-8a17-4d54-8ccb-992b4341f790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:31.811971 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-3cdc2004-8a17-4d54-8ccb-992b4341f790 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:31.813184 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3cdc2004-8a17-4d54-8ccb-992b4341f790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:31.904585 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 57/99] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:31.935639 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb9d59-2eb9-4c43-b86d-313552afd010 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:32.080721 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb9d59-2eb9-4c43-b86d-313552afd010 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:32.119079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec3b3f-f792-4250-b216-b3dd240cbb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:32.119906 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec3b3f-f792-4250-b216-b3dd240cbb41 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:32.120200 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec3b3f-f792-4250-b216-b3dd240cbb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:32.120487 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec3b3f-f792-4250-b216-b3dd240cbb41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:32.132670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec3b3f-f792-4250-b216-b3dd240cbb41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:32.140887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-45ec3b3f-f792-4250-b216-b3dd240cbb41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:32.140887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:32.140887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:32.140887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:32.140887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:32.140887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:32.140887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:32.140887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:32.140887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:32.140887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:32.140887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:32.140887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:32.140887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:32.140887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:32.140887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:32.140887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:32.141734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:32.141734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:32.141734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:32.141734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:32.141734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:32.141734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:32.141734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:32.141734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:32.141734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:32.141734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:32.141734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:32.141734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:32.141734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:32.141734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:32.141734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:32.141734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:32.142499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:32.142499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:32.142499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:32.142499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:32.142499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:32.142499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:32.142499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:32.142499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:32.142499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:32.142499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:32.142499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:32.142499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:22:32.142499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 43/100] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:32.149965 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc2004-8a17-4d54-8ccb-992b4341f790 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:32.161468 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:32.308085 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc2004-8a17-4d54-8ccb-992b4341f790 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:32.356603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f6604-9a79-484a-99c3-cd3f49844dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:32.357215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f6604-9a79-484a-99c3-cd3f49844dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:32.357509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f6604-9a79-484a-99c3-cd3f49844dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:32.357794 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f6604-9a79-484a-99c3-cd3f49844dd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:32.358556 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f6604-9a79-484a-99c3-cd3f49844dd5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:32.358873 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f6604-9a79-484a-99c3-cd3f49844dd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:32.360291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f6604-9a79-484a-99c3-cd3f49844dd5 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:32.374570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 58/101] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:32.380202 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec3b3f-f792-4250-b216-b3dd240cbb41 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:32.419294 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:32.533843 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec3b3f-f792-4250-b216-b3dd240cbb41 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:32.574514 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2608b-d1e7-49cd-bc5d-2f3d55c8a5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:32.575684 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2608b-d1e7-49cd-bc5d-2f3d55c8a5bc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:32.575978 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2608b-d1e7-49cd-bc5d-2f3d55c8a5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:32.576264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2608b-d1e7-49cd-bc5d-2f3d55c8a5bc None admin] PATH_INFO: `/v3/users/ae81942c6123431c8ac728c83a3710b6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:32.577527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f2608b-d1e7-49cd-bc5d-2f3d55c8a5bc None admin] RBAC: Authorizing `identity:get_user(user_id=ae81942c6123431c8ac728c83a3710b6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:32.579010 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f2608b-d1e7-49cd-bc5d-2f3d55c8a5bc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:32.580842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 44/102] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:32 2022] GET /identity/v3/users/ae81942c6123431c8ac728c83a3710b6 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:32.587124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f6604-9a79-484a-99c3-cd3f49844dd5 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:32.591520 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cb097-acc8-4341-8e00-1d49c66aa0d3 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:32.592350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cb097-acc8-4341-8e00-1d49c66aa0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:32.592864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cb097-acc8-4341-8e00-1d49c66aa0d3 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:32.594463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 45/103] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:32.600346 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092c038-31d4-414c-a869-9e358659a0f5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:32.601808 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092c038-31d4-414c-a869-9e358659a0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:32.602292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092c038-31d4-414c-a869-9e358659a0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:32.621201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e092c038-31d4-414c-a869-9e358659a0f5 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:32.622734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e092c038-31d4-414c-a869-9e358659a0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:32.727279 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:32.753071 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 46/104] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:22:32.758944 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99b661-b331-4342-b880-5d09e5ecc433 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:32.759679 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99b661-b331-4342-b880-5d09e5ecc433 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:32.760199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99b661-b331-4342-b880-5d09e5ecc433 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:32.761794 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 47/105] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:32.767406 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24169a34-9a06-49d0-a7f1-14d373bf89af None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:32.767977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24169a34-9a06-49d0-a7f1-14d373bf89af None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:32.768485 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24169a34-9a06-49d0-a7f1-14d373bf89af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:32.778337 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f6604-9a79-484a-99c3-cd3f49844dd5 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:32.808628 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-24169a34-9a06-49d0-a7f1-14d373bf89af None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:32.810762 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-24169a34-9a06-49d0-a7f1-14d373bf89af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:32.826233 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:32.842731 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da310cf-57c0-4213-9ad3-6405752ed2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:32.844471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da310cf-57c0-4213-9ad3-6405752ed2bc None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:32.844900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da310cf-57c0-4213-9ad3-6405752ed2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:32.846859 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da310cf-57c0-4213-9ad3-6405752ed2bc None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:32.851749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da310cf-57c0-4213-9ad3-6405752ed2bc None admin] RBAC: Authorizing `identity:get_project(project_id=53390afb4a404df79ab2d9ae31e8d058)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:32.853448 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da310cf-57c0-4213-9ad3-6405752ed2bc None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:32.857195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 59/106] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:32 2022] GET /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058 => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:32.862504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da310cf-57c0-4213-9ad3-6405752ed2bc None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:32.916607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 48/107] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:22:32.917871 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24169a34-9a06-49d0-a7f1-14d373bf89af None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:33.039156 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da310cf-57c0-4213-9ad3-6405752ed2bc None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:33.060508 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24169a34-9a06-49d0-a7f1-14d373bf89af None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:33.090245 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b72aa9-dcbd-45df-95eb-734cc1d9a621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:33.091623 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b72aa9-dcbd-45df-95eb-734cc1d9a621 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:33.092500 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b72aa9-dcbd-45df-95eb-734cc1d9a621 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:33.093108 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b72aa9-dcbd-45df-95eb-734cc1d9a621 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:33.095459 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:33.098133 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6974b8-c139-4c9a-a43e-16530d248e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:33.099233 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6974b8-c139-4c9a-a43e-16530d248e56 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:33.100807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6974b8-c139-4c9a-a43e-16530d248e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:33.100807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6974b8-c139-4c9a-a43e-16530d248e56 None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058/users/ae81942c6123431c8ac728c83a3710b6/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:33.103450 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6974b8-c139-4c9a-a43e-16530d248e56 None admin] RBAC: Authorizing `identity:create_grant(project_id=53390afb4a404df79ab2d9ae31e8d058, user_id=ae81942c6123431c8ac728c83a3710b6, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:33.105212 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6974b8-c139-4c9a-a43e-16530d248e56 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:33.107076 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b72aa9-dcbd-45df-95eb-734cc1d9a621 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:33.117003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-48b72aa9-dcbd-45df-95eb-734cc1d9a621 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:22:33.117003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:33.117003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:33.117003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:33.117003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:33.117003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:33.117003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:33.117003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:33.117003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:33.117003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:33.117003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:33.117003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:33.117003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:33.117003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:33.117003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:33.117003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:33.118169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:33.118169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:33.118169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:33.118169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:33.118169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:33.118169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:33.118169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:33.118169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:33.118169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:33.118169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:33.118169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:33.118169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:33.118169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:33.118169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:33.118169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:33.118169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:33.118970 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:33.118970 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:33.118970 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:33.118970 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:33.118970 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:33.118970 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:33.118970 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:33.118970 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:33.118970 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:33.118970 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:33.118970 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:22:33.118970 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:33.121867 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 60/108] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:22:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:33.123372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a695bd4-12e5-45e7-ac91-f916dee805ed None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:33.123825 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a695bd4-12e5-45e7-ac91-f916dee805ed None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:33.124283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a695bd4-12e5-45e7-ac91-f916dee805ed None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:33.126000 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 61/109] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:33.126868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a695bd4-12e5-45e7-ac91-f916dee805ed None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:33.128788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 49/110] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:22:32 2022] PUT /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058/users/ae81942c6123431c8ac728c83a3710b6/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:22:33.135057 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6974b8-c139-4c9a-a43e-16530d248e56 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:33.135924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:33.269050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6974b8-c139-4c9a-a43e-16530d248e56 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:33.294082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a695bd4-12e5-45e7-ac91-f916dee805ed None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:33.317897 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e6805-65f6-4160-a371-4b6203da8524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:33.318593 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e6805-65f6-4160-a371-4b6203da8524 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:33.318889 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e6805-65f6-4160-a371-4b6203da8524 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:33.319214 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e6805-65f6-4160-a371-4b6203da8524 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:33.321712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e6805-65f6-4160-a371-4b6203da8524 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:33.322061 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e6805-65f6-4160-a371-4b6203da8524 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:33.323993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e6805-65f6-4160-a371-4b6203da8524 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:33.335237 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 50/111] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:33.336644 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c356e6-2873-41da-9f4b-bd5a34b86cd7 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:33.336953 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c356e6-2873-41da-9f4b-bd5a34b86cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:33.337246 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c356e6-2873-41da-9f4b-bd5a34b86cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:33.346215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb35224-7881-44dd-a8ed-ffa82aac0914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:33.347060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb35224-7881-44dd-a8ed-ffa82aac0914 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:33.347356 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb35224-7881-44dd-a8ed-ffa82aac0914 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:33.347647 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb35224-7881-44dd-a8ed-ffa82aac0914 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:33.358246 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb35224-7881-44dd-a8ed-ffa82aac0914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:33.358651 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-54c356e6-2873-41da-9f4b-bd5a34b86cd7 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:33.359925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-54c356e6-2873-41da-9f4b-bd5a34b86cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:33.366223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-8bb35224-7881-44dd-a8ed-ffa82aac0914 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:22:33.366223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:33.366223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:33.366223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:33.366223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:33.366223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:33.366223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:33.366223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:33.366223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:33.366223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:33.366223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:33.366223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:33.366223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:33.366223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:33.366223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:33.366223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:33.367635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:33.367635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:33.367635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:33.367635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:33.367635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:33.367635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:33.367635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:33.367635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:33.367635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:33.367635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:33.367635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:33.367635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:33.367635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:33.367635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:33.367635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:33.367635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:33.368393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:33.368393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:33.368393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:33.368393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:33.368393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:33.368393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:33.368393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:33.368393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:33.368393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:33.368393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:33.368393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:22:33.368393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:33.368393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 62/112] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:33.369874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bba8f-4add-4f49-95e9-427b2f07ff4f None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:33.369874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bba8f-4add-4f49-95e9-427b2f07ff4f None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:33.370125 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bba8f-4add-4f49-95e9-427b2f07ff4f None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:33.371481 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 63/113] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:33.372844 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbcc31-fb96-4f67-9db5-21371fe5f64f None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:33.373302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbcc31-fb96-4f67-9db5-21371fe5f64f None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:33.373588 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dbcc31-fb96-4f67-9db5-21371fe5f64f None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:33.375030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 64/114] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:33.379578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbcc31-fb96-4f67-9db5-21371fe5f64f None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:33.428526 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:33.437753 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 51/115] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:33.438461 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c356e6-2873-41da-9f4b-bd5a34b86cd7 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:33.455258 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:33.512702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dbcc31-fb96-4f67-9db5-21371fe5f64f None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:33.545208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c356e6-2873-41da-9f4b-bd5a34b86cd7 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:33.552248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89b17d-a8e3-4ee4-8386-ab8e0007b9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:33.553539 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89b17d-a8e3-4ee4-8386-ab8e0007b9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:33.553832 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89b17d-a8e3-4ee4-8386-ab8e0007b9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:33.554160 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89b17d-a8e3-4ee4-8386-ab8e0007b9d6 None admin] PATH_INFO: `/v3/users/c3c5fce3a5074734811c3407aac7fd6b` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:33.555763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89b17d-a8e3-4ee4-8386-ab8e0007b9d6 None admin] RBAC: Authorizing `identity:get_user(user_id=c3c5fce3a5074734811c3407aac7fd6b)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:33.557574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89b17d-a8e3-4ee4-8386-ab8e0007b9d6 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:33.559239 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 65/116] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:33 2022] GET /identity/v3/users/c3c5fce3a5074734811c3407aac7fd6b => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:33.560463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830dbba-5a7f-4ee4-9b93-eedac8a84e2a None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:33.560761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830dbba-5a7f-4ee4-9b93-eedac8a84e2a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:33.561053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830dbba-5a7f-4ee4-9b93-eedac8a84e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:33.578333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-5830dbba-5a7f-4ee4-9b93-eedac8a84e2a None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:33.579567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-5830dbba-5a7f-4ee4-9b93-eedac8a84e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:33.587450 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60488bbc-2261-4d01-a968-a98310581a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:33.588357 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60488bbc-2261-4d01-a968-a98310581a85 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:33.588835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60488bbc-2261-4d01-a968-a98310581a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:33.589287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60488bbc-2261-4d01-a968-a98310581a85 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:33.590228 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60488bbc-2261-4d01-a968-a98310581a85 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:33.590543 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60488bbc-2261-4d01-a968-a98310581a85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:33.591966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60488bbc-2261-4d01-a968-a98310581a85 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:33.604983 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 52/117] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:33.604983 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5ae22-e0c8-4593-b503-73ed7bdf790d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:33.604983 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5ae22-e0c8-4593-b503-73ed7bdf790d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:33.604983 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5ae22-e0c8-4593-b503-73ed7bdf790d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:33.623860 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-96b5ae22-e0c8-4593-b503-73ed7bdf790d None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:33.625752 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-96b5ae22-e0c8-4593-b503-73ed7bdf790d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:33.654161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 66/118] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:33.654527 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830dbba-5a7f-4ee4-9b93-eedac8a84e2a None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:33.700626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:33.722949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 53/119] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:33.722949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5ae22-e0c8-4593-b503-73ed7bdf790d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:33.735708 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:33.786095 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830dbba-5a7f-4ee4-9b93-eedac8a84e2a None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:33.824815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed57ce-655b-4ae7-b3c6-4907a7d5a9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:33.825982 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed57ce-655b-4ae7-b3c6-4907a7d5a9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:33.826390 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed57ce-655b-4ae7-b3c6-4907a7d5a9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:33.826601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed57ce-655b-4ae7-b3c6-4907a7d5a9c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:33.837825 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed57ce-655b-4ae7-b3c6-4907a7d5a9c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:33.839379 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5ae22-e0c8-4593-b503-73ed7bdf790d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:33.846474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-d1ed57ce-655b-4ae7-b3c6-4907a7d5a9c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:33.846474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:33.846474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:33.846474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:33.846474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:33.846474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:33.846474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:33.846474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:33.846474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:33.846474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:33.846474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:33.846474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:33.846474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:33.846474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:33.846474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:33.846474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:33.847669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:33.847669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:33.847669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:33.847669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:33.847669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:33.847669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:33.847669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:33.847669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:33.847669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:33.847669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:33.847669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:33.847669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:33.847669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:33.847669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:33.847669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:33.847669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:33.848434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:33.848434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:33.848434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:33.848434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:33.848434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:33.848434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:33.848434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:33.848434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:33.848434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:33.848434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:33.848434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:33.848434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:33.848434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 67/120] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:33.849079 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed57ce-655b-4ae7-b3c6-4907a7d5a9c7 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:33.879526 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3db3c-c9ef-4c19-834c-54267142d379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:33.880287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3db3c-c9ef-4c19-834c-54267142d379 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:33.880571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3db3c-c9ef-4c19-834c-54267142d379 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:33.880998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3db3c-c9ef-4c19-834c-54267142d379 None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:33.882037 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3db3c-c9ef-4c19-834c-54267142d379 None admin] RBAC: Authorizing `identity:get_project(project_id=53390afb4a404df79ab2d9ae31e8d058)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:33.884185 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3db3c-c9ef-4c19-834c-54267142d379 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:33.886058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 54/121] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:33 2022] GET /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:33.886738 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3db3c-c9ef-4c19-834c-54267142d379 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:33.950255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:33.970180 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed57ce-655b-4ae7-b3c6-4907a7d5a9c7 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:33.985026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:33.998753 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3db3c-c9ef-4c19-834c-54267142d379 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:34.008221 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5a0d4-f147-470d-b38f-5dbe79ab09bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:34.009107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5a0d4-f147-470d-b38f-5dbe79ab09bb None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:34.009397 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5a0d4-f147-470d-b38f-5dbe79ab09bb None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:34.009680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5a0d4-f147-470d-b38f-5dbe79ab09bb None admin] PATH_INFO: `/v3/users/fb14ca6f324b4b46b6fbdb67d0fecb95` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:34.024169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5a0d4-f147-470d-b38f-5dbe79ab09bb None admin] RBAC: Authorizing `identity:get_user(user_id=fb14ca6f324b4b46b6fbdb67d0fecb95)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:34.026277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5a0d4-f147-470d-b38f-5dbe79ab09bb None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:34.028178 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 68/122] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:33 2022] GET /identity/v3/users/fb14ca6f324b4b46b6fbdb67d0fecb95 => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:34.033079 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5a0d4-f147-470d-b38f-5dbe79ab09bb None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:34.035409 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa03a51-30c3-4d57-a815-c938d9ed04b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:34.036210 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa03a51-30c3-4d57-a815-c938d9ed04b6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:34.036505 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa03a51-30c3-4d57-a815-c938d9ed04b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:34.036790 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa03a51-30c3-4d57-a815-c938d9ed04b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:34.037978 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa03a51-30c3-4d57-a815-c938d9ed04b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:34.039870 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa03a51-30c3-4d57-a815-c938d9ed04b6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:34.041622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 55/123] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:22:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:34.046805 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa03a51-30c3-4d57-a815-c938d9ed04b6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:34.138475 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5a0d4-f147-470d-b38f-5dbe79ab09bb None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:34.157844 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa03a51-30c3-4d57-a815-c938d9ed04b6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:34.173809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6c098-1116-43d1-b1cc-1fddbde7726d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:34.174633 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6c098-1116-43d1-b1cc-1fddbde7726d None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:34.174931 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6c098-1116-43d1-b1cc-1fddbde7726d None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:34.175227 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6c098-1116-43d1-b1cc-1fddbde7726d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:34.182406 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:34.186897 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6c098-1116-43d1-b1cc-1fddbde7726d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:34.194378 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7e305-925b-478a-a21a-b58b9ba5483c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:34.194910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-49a6c098-1116-43d1-b1cc-1fddbde7726d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:22:34.194910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:34.194910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:34.194910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:34.194910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:34.194910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:34.194910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:34.194910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:34.194910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:34.194910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:34.194910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:34.194910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:34.194910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:34.194910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:34.194910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:34.194910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:34.195928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:34.195928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:34.195928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:34.195928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:34.195928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:34.195928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:34.195928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:34.195928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:34.195928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:34.195928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:34.195928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:34.195928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:34.195928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:34.195928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:34.195928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:34.195928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:34.196703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:34.196703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:34.196703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:34.196703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:34.196703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:34.196703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:34.196703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:34.196703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:34.196703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:34.196703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:34.196703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:22:34.196703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:34.197285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7e305-925b-478a-a21a-b58b9ba5483c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:34.197285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7e305-925b-478a-a21a-b58b9ba5483c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:34.197285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7e305-925b-478a-a21a-b58b9ba5483c None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:34.197466 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 69/124] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:22:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:34.197535 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a7e305-925b-478a-a21a-b58b9ba5483c None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:34.197535 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a7e305-925b-478a-a21a-b58b9ba5483c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:34.198542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a7e305-925b-478a-a21a-b58b9ba5483c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:34.201686 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6c098-1116-43d1-b1cc-1fddbde7726d None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:34.213177 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 56/125] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:34.213177 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7e305-925b-478a-a21a-b58b9ba5483c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:34.236727 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:34.337072 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6c098-1116-43d1-b1cc-1fddbde7726d None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:34.338044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7e305-925b-478a-a21a-b58b9ba5483c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:34.373419 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f21e86-9216-423c-9a90-af4cb7699a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:34.374307 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f21e86-9216-423c-9a90-af4cb7699a21 None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:34.374589 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f21e86-9216-423c-9a90-af4cb7699a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:34.374878 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f21e86-9216-423c-9a90-af4cb7699a21 None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058/users/c3c5fce3a5074734811c3407aac7fd6b/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:34.375993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f2adf-4c47-4570-9039-49e94ee4fe06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:34.376599 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f2adf-4c47-4570-9039-49e94ee4fe06 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:34.376894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f2adf-4c47-4570-9039-49e94ee4fe06 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:34.376983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f21e86-9216-423c-9a90-af4cb7699a21 None admin] RBAC: Authorizing `identity:create_grant(project_id=53390afb4a404df79ab2d9ae31e8d058, user_id=c3c5fce3a5074734811c3407aac7fd6b, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:34.377184 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f2adf-4c47-4570-9039-49e94ee4fe06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:34.378387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f21e86-9216-423c-9a90-af4cb7699a21 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:34.393749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 70/126] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:22:34 2022] PUT /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058/users/c3c5fce3a5074734811c3407aac7fd6b/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:22:34.394825 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f21e86-9216-423c-9a90-af4cb7699a21 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:34.454585 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:34.471882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f2adf-4c47-4570-9039-49e94ee4fe06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:34.473525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f2adf-4c47-4570-9039-49e94ee4fe06 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:34.532910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:34.550340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f21e86-9216-423c-9a90-af4cb7699a21 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:34.598884 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 57/127] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:22:34.605143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f2adf-4c47-4570-9039-49e94ee4fe06 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:34.605776 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c679b-5199-44ed-b296-201796587117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:34.606470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c679b-5199-44ed-b296-201796587117 None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:34.606761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c679b-5199-44ed-b296-201796587117 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:34.607122 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92c679b-5199-44ed-b296-201796587117 None admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:34.607855 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c679b-5199-44ed-b296-201796587117 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:34.609324 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92c679b-5199-44ed-b296-201796587117 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:34.669332 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 71/128] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 22:22:34 2022] POST /identity/v3/projects => generated 302 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:22:34.670412 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c679b-5199-44ed-b296-201796587117 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:34.813258 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92c679b-5199-44ed-b296-201796587117 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:34.821648 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:34.862093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f8fa1-d261-40e8-abdb-066007fa0030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:34.862093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f8fa1-d261-40e8-abdb-066007fa0030 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:34.862093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f8fa1-d261-40e8-abdb-066007fa0030 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:34.862093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f8fa1-d261-40e8-abdb-066007fa0030 None admin] PATH_INFO: `/v3/users/ae81942c6123431c8ac728c83a3710b6` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:34.862806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f8fa1-d261-40e8-abdb-066007fa0030 None admin] RBAC: Authorizing `identity:get_user(user_id=ae81942c6123431c8ac728c83a3710b6)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:34.862806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f8fa1-d261-40e8-abdb-066007fa0030 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:34.862806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 72/129] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:34 2022] GET /identity/v3/users/ae81942c6123431c8ac728c83a3710b6 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:34.863029 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f8fa1-d261-40e8-abdb-066007fa0030 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:34.863497 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f2adf-4c47-4570-9039-49e94ee4fe06 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:34.883529 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:34.963097 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85959c-9ecd-47ba-8d9b-3f62060f06d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:34.963913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85959c-9ecd-47ba-8d9b-3f62060f06d9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:34.964204 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85959c-9ecd-47ba-8d9b-3f62060f06d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:34.964486 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85959c-9ecd-47ba-8d9b-3f62060f06d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:34.965271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85959c-9ecd-47ba-8d9b-3f62060f06d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:34.965587 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85959c-9ecd-47ba-8d9b-3f62060f06d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:34.966989 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85959c-9ecd-47ba-8d9b-3f62060f06d9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:34.981643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 58/130] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:22:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:34.982539 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85959c-9ecd-47ba-8d9b-3f62060f06d9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:35.033965 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f8fa1-d261-40e8-abdb-066007fa0030 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:35.118331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4481b69-54d1-474b-9dd2-3232ed12a26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:35.120114 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4481b69-54d1-474b-9dd2-3232ed12a26a None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:35.120407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4481b69-54d1-474b-9dd2-3232ed12a26a None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:35.120693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4481b69-54d1-474b-9dd2-3232ed12a26a None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:35.129842 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4481b69-54d1-474b-9dd2-3232ed12a26a None admin] RBAC: Authorizing `identity:get_project(project_id=53390afb4a404df79ab2d9ae31e8d058)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:35.131404 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4481b69-54d1-474b-9dd2-3232ed12a26a None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:35.134018 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 73/131] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:34 2022] GET /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058 => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:35.138465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4481b69-54d1-474b-9dd2-3232ed12a26a None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:35.159680 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85959c-9ecd-47ba-8d9b-3f62060f06d9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:35.211430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6fd40-5326-4c9a-b342-2ea6fee8bad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:35.215834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6fd40-5326-4c9a-b342-2ea6fee8bad8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:35.215834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6fd40-5326-4c9a-b342-2ea6fee8bad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:35.215834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6fd40-5326-4c9a-b342-2ea6fee8bad8 None admin] PATH_INFO: `/v3/projects/ff7c7330c1444db4a0c394169b7517d0` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:35.215834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6fd40-5326-4c9a-b342-2ea6fee8bad8 None admin] RBAC: Authorizing `identity:get_project(project_id=ff7c7330c1444db4a0c394169b7517d0)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:35.216120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6fd40-5326-4c9a-b342-2ea6fee8bad8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:35.218088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 59/132] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:34 2022] GET /identity/v3/projects/ff7c7330c1444db4a0c394169b7517d0 => generated 312 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:35.219972 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6fd40-5326-4c9a-b342-2ea6fee8bad8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:35.252665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:35.298042 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:35.349076 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4481b69-54d1-474b-9dd2-3232ed12a26a None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:35.373361 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6fd40-5326-4c9a-b342-2ea6fee8bad8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:35.401603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3f926-13b0-4f36-8911-67a605974191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:35.402766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3f926-13b0-4f36-8911-67a605974191 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:35.403228 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3f926-13b0-4f36-8911-67a605974191 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:35.403533 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3f926-13b0-4f36-8911-67a605974191 None admin] PATH_INFO: `/v3/users/ae81942c6123431c8ac728c83a3710b6` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:35.405345 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3f926-13b0-4f36-8911-67a605974191 None admin] RBAC: Authorizing `identity:get_user(user_id=ae81942c6123431c8ac728c83a3710b6)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:35.407306 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3f926-13b0-4f36-8911-67a605974191 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:35.411842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154ced4-9577-4ced-85bb-4f422aa176a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:35.412503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154ced4-9577-4ced-85bb-4f422aa176a5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:35.412790 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154ced4-9577-4ced-85bb-4f422aa176a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:35.413072 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154ced4-9577-4ced-85bb-4f422aa176a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:35.413858 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 74/133] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:35 2022] GET /identity/v3/users/ae81942c6123431c8ac728c83a3710b6 => generated 275 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:35.414046 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154ced4-9577-4ced-85bb-4f422aa176a5 None admin] RBAC: Adding query filter params (role.id=ff3357541f744b6094e8e5fb5a79aa32, scope.project.id=53390afb4a404df79ab2d9ae31e8d058, user.id=fb14ca6f324b4b46b6fbdb67d0fecb95) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:35.414343 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154ced4-9577-4ced-85bb-4f422aa176a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:35.414695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3f926-13b0-4f36-8911-67a605974191 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:35.415913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154ced4-9577-4ced-85bb-4f422aa176a5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:35.422384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 60/134] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:22:35 2022] GET /identity/v3/role_assignments?user.id=fb14ca6f324b4b46b6fbdb67d0fecb95&scope.project.id=53390afb4a404df79ab2d9ae31e8d058&role.id=ff3357541f744b6094e8e5fb5a79aa32 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:35.432183 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09084e-9453-43b6-83ad-8f1cc3bb0f4c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:35.432503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09084e-9453-43b6-83ad-8f1cc3bb0f4c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:35.432864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09084e-9453-43b6-83ad-8f1cc3bb0f4c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:35.434585 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 61/135] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:35 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:35.435199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09084e-9453-43b6-83ad-8f1cc3bb0f4c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:35.545287 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3f926-13b0-4f36-8911-67a605974191 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:35.552004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:35.575966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09084e-9453-43b6-83ad-8f1cc3bb0f4c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:35.632428 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31fe31-f8bd-41e3-928b-da5706081744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:35.633723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31fe31-f8bd-41e3-928b-da5706081744 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:35.634226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31fe31-f8bd-41e3-928b-da5706081744 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:35.634720 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31fe31-f8bd-41e3-928b-da5706081744 None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:35.636233 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31fe31-f8bd-41e3-928b-da5706081744 None admin] RBAC: Authorizing `identity:get_project(project_id=53390afb4a404df79ab2d9ae31e8d058)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:35.637928 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31fe31-f8bd-41e3-928b-da5706081744 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:35.639718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2537d91-ef53-4309-97d6-00756e3c2685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:35.640555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 62/136] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:35 2022] GET /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:35.640756 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2537d91-ef53-4309-97d6-00756e3c2685 None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:35.641018 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2537d91-ef53-4309-97d6-00756e3c2685 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:35.641310 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2537d91-ef53-4309-97d6-00756e3c2685 None admin] PATH_INFO: `/v3/projects/ff7c7330c1444db4a0c394169b7517d0/users/ae81942c6123431c8ac728c83a3710b6/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:35.647192 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d8259-7bc5-4ca5-9569-6373470dfcfb None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:35.647331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2537d91-ef53-4309-97d6-00756e3c2685 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff7c7330c1444db4a0c394169b7517d0, user_id=ae81942c6123431c8ac728c83a3710b6, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:35.647331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2537d91-ef53-4309-97d6-00756e3c2685 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:35.648556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d8259-7bc5-4ca5-9569-6373470dfcfb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:35.649062 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d8259-7bc5-4ca5-9569-6373470dfcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:35.666471 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-993d8259-7bc5-4ca5-9569-6373470dfcfb None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:35.668015 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-993d8259-7bc5-4ca5-9569-6373470dfcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:35.678950 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 75/137] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:22:35 2022] PUT /identity/v3/projects/ff7c7330c1444db4a0c394169b7517d0/users/ae81942c6123431c8ac728c83a3710b6/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:22:35.682628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2537d91-ef53-4309-97d6-00756e3c2685 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:35.695572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:35.753479 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 63/138] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:35.794012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d8259-7bc5-4ca5-9569-6373470dfcfb None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:35.879186 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2537d91-ef53-4309-97d6-00756e3c2685 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:35.949968 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651b374-6ea8-47dd-874c-3b04a8f6bb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:35.951336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651b374-6ea8-47dd-874c-3b04a8f6bb16 None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:35.951872 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651b374-6ea8-47dd-874c-3b04a8f6bb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:35.952360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651b374-6ea8-47dd-874c-3b04a8f6bb16 None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058/users/ae81942c6123431c8ac728c83a3710b6/roles/61397d4a1ccd4a8ca479f98186ed489b` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:35.961772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651b374-6ea8-47dd-874c-3b04a8f6bb16 None admin] RBAC: Authorizing `identity:create_grant(project_id=53390afb4a404df79ab2d9ae31e8d058, user_id=ae81942c6123431c8ac728c83a3710b6, role_id=61397d4a1ccd4a8ca479f98186ed489b)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:35.963726 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651b374-6ea8-47dd-874c-3b04a8f6bb16 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:35.978960 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:35.994825 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 76/139] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:22:35 2022] PUT /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058/users/ae81942c6123431c8ac728c83a3710b6/roles/61397d4a1ccd4a8ca479f98186ed489b => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:22:36.027134 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d8259-7bc5-4ca5-9569-6373470dfcfb None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:36.074734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbff908-d3cc-4681-964f-07f878fc2466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:36.075990 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbff908-d3cc-4681-964f-07f878fc2466 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:36.076498 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbff908-d3cc-4681-964f-07f878fc2466 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:36.077217 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbff908-d3cc-4681-964f-07f878fc2466 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:36.093603 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbff908-d3cc-4681-964f-07f878fc2466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:36.105583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-fdbff908-d3cc-4681-964f-07f878fc2466 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:36.105583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:36.105583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:36.105583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:36.105583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:36.105583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:36.105583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:36.105583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:36.105583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:36.105583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:36.105583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:36.105583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:36.105583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:36.105583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:36.105583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:36.105583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:36.106523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:36.106523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:36.106523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:36.106523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:36.106523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:36.106523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:36.106523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:36.106523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:36.106523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:36.106523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:36.106523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:36.106523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:36.106523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:36.106523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:36.106523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:36.106523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:36.107305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:36.107305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:36.107305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:36.107305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:36.107305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:36.107305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:36.107305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:36.107305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:36.107305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:36.107305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:36.107305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:36.107305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:22:36.109703 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 64/140] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:36.115743 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651b374-6ea8-47dd-874c-3b04a8f6bb16 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:36.193696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:36.238114 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651b374-6ea8-47dd-874c-3b04a8f6bb16 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:36.286567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85622e04-c12a-47f1-b698-6ac8f86e9f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:36.287364 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85622e04-c12a-47f1-b698-6ac8f86e9f0e None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:36.287659 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85622e04-c12a-47f1-b698-6ac8f86e9f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:36.287948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85622e04-c12a-47f1-b698-6ac8f86e9f0e None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:36.288891 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85622e04-c12a-47f1-b698-6ac8f86e9f0e None admin] RBAC: Adding query filter params (name=member) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:36.289210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85622e04-c12a-47f1-b698-6ac8f86e9f0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:36.290623 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85622e04-c12a-47f1-b698-6ac8f86e9f0e None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:36.299949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 77/141] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:36.308737 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbff908-d3cc-4681-964f-07f878fc2466 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:36.424139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbff908-d3cc-4681-964f-07f878fc2466 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:36.462722 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f9b01-1b92-43a2-9600-f8b370ff11ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:36.463568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f9b01-1b92-43a2-9600-f8b370ff11ee None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:36.463862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f9b01-1b92-43a2-9600-f8b370ff11ee None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:36.464217 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f9b01-1b92-43a2-9600-f8b370ff11ee None admin] PATH_INFO: `/v3/users/ae81942c6123431c8ac728c83a3710b6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:36.465515 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f9b01-1b92-43a2-9600-f8b370ff11ee None admin] RBAC: Authorizing `identity:get_user(user_id=ae81942c6123431c8ac728c83a3710b6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:36.467019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f9b01-1b92-43a2-9600-f8b370ff11ee None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:36.468668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 65/142] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:36 2022] GET /identity/v3/users/ae81942c6123431c8ac728c83a3710b6 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:36.474313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85622e04-c12a-47f1-b698-6ac8f86e9f0e None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:36.592657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85622e04-c12a-47f1-b698-6ac8f86e9f0e None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:36.620102 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:36.635277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1003c-b1bc-49f6-912d-d6f962065e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:36.635955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1003c-b1bc-49f6-912d-d6f962065e79 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:36.636255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1003c-b1bc-49f6-912d-d6f962065e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:36.636550 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1003c-b1bc-49f6-912d-d6f962065e79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:36.718613 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1003c-b1bc-49f6-912d-d6f962065e79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:36.720185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1003c-b1bc-49f6-912d-d6f962065e79 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:36.829496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 78/143] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:22:36.836110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f9b01-1b92-43a2-9600-f8b370ff11ee None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:36.865616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:36.964447 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f9b01-1b92-43a2-9600-f8b370ff11ee None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:37.011115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8067de4-bc57-42f0-b417-82879cef3e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:37.012659 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8067de4-bc57-42f0-b417-82879cef3e87 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:37.013136 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8067de4-bc57-42f0-b417-82879cef3e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:37.013566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8067de4-bc57-42f0-b417-82879cef3e87 None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:37.015930 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8067de4-bc57-42f0-b417-82879cef3e87 None admin] RBAC: Authorizing `identity:get_project(project_id=53390afb4a404df79ab2d9ae31e8d058)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:37.017625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8067de4-bc57-42f0-b417-82879cef3e87 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:37.019534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 66/144] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:36 2022] GET /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:37.026407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1003c-b1bc-49f6-912d-d6f962065e79 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:37.063396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:37.173103 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1003c-b1bc-49f6-912d-d6f962065e79 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:37.209064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32baaf5d-c878-4f7a-b58c-3734a6d0933d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:37.209582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32baaf5d-c878-4f7a-b58c-3734a6d0933d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:37.210708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32baaf5d-c878-4f7a-b58c-3734a6d0933d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:37.213002 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 67/145] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:37.218586 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fbab0-af10-4059-8a25-00dcdd75dd1d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:37.219303 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fbab0-af10-4059-8a25-00dcdd75dd1d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:37.220084 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fbab0-af10-4059-8a25-00dcdd75dd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:37.238590 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60bb40-8bab-4c8b-a3b8-0c23980dfed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:37.239297 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60bb40-8bab-4c8b-a3b8-0c23980dfed5 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:37.239583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60bb40-8bab-4c8b-a3b8-0c23980dfed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:37.239860 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60bb40-8bab-4c8b-a3b8-0c23980dfed5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:37.240706 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b60bb40-8bab-4c8b-a3b8-0c23980dfed5 None admin] RBAC: Adding query filter params (role.id=1baba2239751440eb25af0d0fd2d072d, scope.project.id=53390afb4a404df79ab2d9ae31e8d058, user.id=ae81942c6123431c8ac728c83a3710b6) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:37.241830 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b60bb40-8bab-4c8b-a3b8-0c23980dfed5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:37.242662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b60bb40-8bab-4c8b-a3b8-0c23980dfed5 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:37.260255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-107fbab0-af10-4059-8a25-00dcdd75dd1d None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:37.261823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-107fbab0-af10-4059-8a25-00dcdd75dd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:37.265756 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 79/146] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:22:37 2022] GET /identity/v3/role_assignments?user.id=ae81942c6123431c8ac728c83a3710b6&scope.project.id=53390afb4a404df79ab2d9ae31e8d058&role.id=1baba2239751440eb25af0d0fd2d072d => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:37.369767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 68/147] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:37.414138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60bb40-8bab-4c8b-a3b8-0c23980dfed5 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:37.558581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:37.567111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60bb40-8bab-4c8b-a3b8-0c23980dfed5 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:37.604992 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9a75d-7016-499b-ad49-1041806351be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:37.605792 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9a75d-7016-499b-ad49-1041806351be None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:37.606079 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9a75d-7016-499b-ad49-1041806351be None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:37.606356 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9a75d-7016-499b-ad49-1041806351be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:37.617604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e9a75d-7016-499b-ad49-1041806351be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:37.628041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-19e9a75d-7016-499b-ad49-1041806351be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:22:37.628041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:37.628041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:37.628041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:37.628041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:37.628041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:37.628041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:37.628041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:37.628041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:37.628041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:37.628041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:37.628041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:37.628041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:37.628041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:37.628041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:37.628041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:37.628949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:37.628949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:37.628949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:37.628949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:37.628949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:37.628949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:37.628949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:37.628949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:37.628949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:37.628949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:37.628949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:37.628949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:37.628949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:37.628949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:37.628949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:37.628949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:37.629719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:37.629719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:37.629719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:37.629719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:37.629719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:37.629719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:37.629719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:37.629719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:37.629719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:37.629719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:37.629719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:22:37.629719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:37.629719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 80/148] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:22:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:37.637987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fbab0-af10-4059-8a25-00dcdd75dd1d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:37.651739 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:37.746999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fbab0-af10-4059-8a25-00dcdd75dd1d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:37.791113 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e06024-2186-40ac-9899-08c25cbd28fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:37.791793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e06024-2186-40ac-9899-08c25cbd28fd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:37.792083 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e06024-2186-40ac-9899-08c25cbd28fd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:37.792365 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e06024-2186-40ac-9899-08c25cbd28fd None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:37.793120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e06024-2186-40ac-9899-08c25cbd28fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:37.793433 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e06024-2186-40ac-9899-08c25cbd28fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:37.794823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e06024-2186-40ac-9899-08c25cbd28fd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:37.806086 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 69/149] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:37.811560 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9a75d-7016-499b-ad49-1041806351be None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:37.846907 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31103fb-9695-40b3-87c6-e2e74245e0ee None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:37.847249 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31103fb-9695-40b3-87c6-e2e74245e0ee None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:37.847494 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31103fb-9695-40b3-87c6-e2e74245e0ee None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:37.848999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 70/150] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:22:37.885378 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cf82c-1dd0-4e89-aa9e-c0c2cff5fcbe None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:37.885689 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cf82c-1dd0-4e89-aa9e-c0c2cff5fcbe None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:37.885953 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cf82c-1dd0-4e89-aa9e-c0c2cff5fcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:37.918305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4e1cf82c-1dd0-4e89-aa9e-c0c2cff5fcbe None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:37.919621 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4e1cf82c-1dd0-4e89-aa9e-c0c2cff5fcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:37.958444 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e9a75d-7016-499b-ad49-1041806351be None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:38.019393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:38.025684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f45a43-2401-4161-9336-377d263cecb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:38.026595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f45a43-2401-4161-9336-377d263cecb7 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:38.027161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f45a43-2401-4161-9336-377d263cecb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:38.027161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f45a43-2401-4161-9336-377d263cecb7 None admin] PATH_INFO: `/v3/users/c3c5fce3a5074734811c3407aac7fd6b` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:38.029276 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f45a43-2401-4161-9336-377d263cecb7 None admin] RBAC: Authorizing `identity:get_user(user_id=c3c5fce3a5074734811c3407aac7fd6b)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:38.030785 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f45a43-2401-4161-9336-377d263cecb7 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:38.035144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 81/151] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:37 2022] GET /identity/v3/users/c3c5fce3a5074734811c3407aac7fd6b => generated 247 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:38.036708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 71/152] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:38.038780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cf82c-1dd0-4e89-aa9e-c0c2cff5fcbe None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:38.066398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f45a43-2401-4161-9336-377d263cecb7 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:38.103383 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:38.222239 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f45a43-2401-4161-9336-377d263cecb7 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:38.243079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cf82c-1dd0-4e89-aa9e-c0c2cff5fcbe None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:38.259018 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff59a99-3bae-452c-b750-7e58bb80e7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:38.259299 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff59a99-3bae-452c-b750-7e58bb80e7a2 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:38.260110 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff59a99-3bae-452c-b750-7e58bb80e7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:38.260110 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff59a99-3bae-452c-b750-7e58bb80e7a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:38.261300 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff59a99-3bae-452c-b750-7e58bb80e7a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:38.262628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff59a99-3bae-452c-b750-7e58bb80e7a2 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:38.264218 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 82/153] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:22:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:38.265554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1082f3-629c-4d25-8c4f-39df492db31a None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:38.266119 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1082f3-629c-4d25-8c4f-39df492db31a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:38.266119 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1082f3-629c-4d25-8c4f-39df492db31a None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:38.267479 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 83/154] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:38.272038 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1082f3-629c-4d25-8c4f-39df492db31a None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:38.286141 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f92fb-fc94-4c59-b0e9-b720f60b7fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:38.286798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f92fb-fc94-4c59-b0e9-b720f60b7fda None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:38.287221 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f92fb-fc94-4c59-b0e9-b720f60b7fda None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:38.287510 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f92fb-fc94-4c59-b0e9-b720f60b7fda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:38.362642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f92fb-fc94-4c59-b0e9-b720f60b7fda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:38.364206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f92fb-fc94-4c59-b0e9-b720f60b7fda None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:38.369204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:38.385989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1082f3-629c-4d25-8c4f-39df492db31a None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:38.387399 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:38.431590 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac130ee-add3-465d-bba3-552232c59b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:38.432275 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac130ee-add3-465d-bba3-552232c59b16 None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:38.432557 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac130ee-add3-465d-bba3-552232c59b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:38.432836 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac130ee-add3-465d-bba3-552232c59b16 None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:38.433552 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac130ee-add3-465d-bba3-552232c59b16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:38.435030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac130ee-add3-465d-bba3-552232c59b16 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:38.459158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 84/155] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 22:22:38 2022] POST /identity/v3/users => generated 283 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:22:38.461277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea870fde-ba22-43e5-a026-fb028158b94a None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:38.461622 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea870fde-ba22-43e5-a026-fb028158b94a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:38.461911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea870fde-ba22-43e5-a026-fb028158b94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:38.479108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 72/156] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 22:22:38.481693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-ea870fde-ba22-43e5-a026-fb028158b94a None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:38.482895 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-ea870fde-ba22-43e5-a026-fb028158b94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:38.484778 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f92fb-fc94-4c59-b0e9-b720f60b7fda None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:38.571298 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 85/157] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:38.595078 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea870fde-ba22-43e5-a026-fb028158b94a None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:38.608401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:38.615041 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f92fb-fc94-4c59-b0e9-b720f60b7fda None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:38.653816 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4465d4-7193-4ca3-a5ce-e86db9b17b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:38.655115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4465d4-7193-4ca3-a5ce-e86db9b17b68 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:38.655625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4465d4-7193-4ca3-a5ce-e86db9b17b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:38.656149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4465d4-7193-4ca3-a5ce-e86db9b17b68 None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:38.657578 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4465d4-7193-4ca3-a5ce-e86db9b17b68 None admin] RBAC: Authorizing `identity:get_project(project_id=53390afb4a404df79ab2d9ae31e8d058)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:38.659398 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4465d4-7193-4ca3-a5ce-e86db9b17b68 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:38.661837 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 73/158] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:38 2022] GET /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:38.672031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4465d4-7193-4ca3-a5ce-e86db9b17b68 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:38.686305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:38.719371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea870fde-ba22-43e5-a026-fb028158b94a None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:38.767756 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45290cb4-639d-400f-b392-ee1477736410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:38.768898 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45290cb4-639d-400f-b392-ee1477736410 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:38.769576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45290cb4-639d-400f-b392-ee1477736410 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:38.770463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45290cb4-639d-400f-b392-ee1477736410 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:38.794166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45290cb4-639d-400f-b392-ee1477736410 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:38.808570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-45290cb4-639d-400f-b392-ee1477736410 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:22:38.808570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:38.808570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:38.808570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:38.808570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:38.808570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:38.808570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:38.808570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:38.808570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:38.808570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:38.808570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:38.808570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:38.808570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:38.808570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:38.808570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:38.808570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:38.809688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:38.809688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:38.809688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:38.809688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:38.809688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:38.809688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:38.809688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:38.809688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:38.809688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:38.809688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:38.809688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:38.809688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:38.809688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:38.809688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:38.809688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:38.809688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:38.810691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:38.810691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:38.810691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:38.810691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:38.810691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:38.810691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:38.810691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:38.810691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:38.810691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:38.810691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:38.810691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:22:38.810691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:38.813225 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 86/159] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:38.819427 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45290cb4-639d-400f-b392-ee1477736410 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:38.839250 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4465d4-7193-4ca3-a5ce-e86db9b17b68 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:38.884945 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b2253-1575-4645-895a-b548845b1322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:38.885926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b2253-1575-4645-895a-b548845b1322 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:38.886408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b2253-1575-4645-895a-b548845b1322 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:38.886897 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b2253-1575-4645-895a-b548845b1322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:38.887977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b2253-1575-4645-895a-b548845b1322 None admin] RBAC: Adding query filter params (role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.project.id=53390afb4a404df79ab2d9ae31e8d058, user.id=c3c5fce3a5074734811c3407aac7fd6b) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:38.888485 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b2253-1575-4645-895a-b548845b1322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:38.890139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b2253-1575-4645-895a-b548845b1322 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:38.897401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 74/160] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:22:38 2022] GET /identity/v3/role_assignments?user.id=c3c5fce3a5074734811c3407aac7fd6b&scope.project.id=53390afb4a404df79ab2d9ae31e8d058&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:38.941331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45290cb4-639d-400f-b392-ee1477736410 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:38.982905 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e809fd-2d2d-4564-aee8-be0654796e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:38.983636 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e809fd-2d2d-4564-aee8-be0654796e14 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:38.983987 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e809fd-2d2d-4564-aee8-be0654796e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:38.984215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e809fd-2d2d-4564-aee8-be0654796e14 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:38.984988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e809fd-2d2d-4564-aee8-be0654796e14 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:38.985305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e809fd-2d2d-4564-aee8-be0654796e14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:38.986766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e809fd-2d2d-4564-aee8-be0654796e14 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:38.988416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:39.003056 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 87/161] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:39.009380 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b2253-1575-4645-895a-b548845b1322 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:39.045848 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:39.092388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c86426-8206-42e8-94cd-bf6be1b167a7 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:39.092875 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c86426-8206-42e8-94cd-bf6be1b167a7 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:39.093050 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c86426-8206-42e8-94cd-bf6be1b167a7 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:39.094886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 88/162] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:39.100115 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb044a-194d-4514-b03a-de6392545359 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:39.100405 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb044a-194d-4514-b03a-de6392545359 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:39.100689 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb044a-194d-4514-b03a-de6392545359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:39.127961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-56eb044a-194d-4514-b03a-de6392545359 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:39.129179 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-56eb044a-194d-4514-b03a-de6392545359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:39.138689 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b2253-1575-4645-895a-b548845b1322 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:39.179511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc39c8-f2c6-4a32-b0a5-07540ab1581d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:39.180762 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc39c8-f2c6-4a32-b0a5-07540ab1581d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:39.181146 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc39c8-f2c6-4a32-b0a5-07540ab1581d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:39.181572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc39c8-f2c6-4a32-b0a5-07540ab1581d None admin] PATH_INFO: `/v3/users/fb14ca6f324b4b46b6fbdb67d0fecb95` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:39.182945 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edc39c8-f2c6-4a32-b0a5-07540ab1581d None admin] RBAC: Authorizing `identity:get_user(user_id=fb14ca6f324b4b46b6fbdb67d0fecb95)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:39.185717 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edc39c8-f2c6-4a32-b0a5-07540ab1581d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:39.187636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 75/163] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:39 2022] GET /identity/v3/users/fb14ca6f324b4b46b6fbdb67d0fecb95 => generated 289 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:39.188737 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f0893-3469-4edf-8adc-1a4bf471633a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:39.189027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f0893-3469-4edf-8adc-1a4bf471633a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:39.189317 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f0893-3469-4edf-8adc-1a4bf471633a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:39.190567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 76/164] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:39.194451 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f0893-3469-4edf-8adc-1a4bf471633a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:39.230316 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 89/165] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:39.231761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05114e72-e132-49d6-84fd-7e86ce55b68e None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:39.232062 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05114e72-e132-49d6-84fd-7e86ce55b68e None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:39.232347 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05114e72-e132-49d6-84fd-7e86ce55b68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:39.249524 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-05114e72-e132-49d6-84fd-7e86ce55b68e None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:39.250834 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-05114e72-e132-49d6-84fd-7e86ce55b68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:39.320522 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:39.328738 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f0893-3469-4edf-8adc-1a4bf471633a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:39.330465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 90/166] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:39.331575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05114e72-e132-49d6-84fd-7e86ce55b68e None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:39.375718 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d55c4-1b67-4b8b-9a4b-dc537556e9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:39.376602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d55c4-1b67-4b8b-9a4b-dc537556e9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:39.376892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d55c4-1b67-4b8b-9a4b-dc537556e9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:39.377348 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d55c4-1b67-4b8b-9a4b-dc537556e9a6 None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:39.379484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d55c4-1b67-4b8b-9a4b-dc537556e9a6 None admin] RBAC: Authorizing `identity:get_project(project_id=53390afb4a404df79ab2d9ae31e8d058)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:39.380961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d55c4-1b67-4b8b-9a4b-dc537556e9a6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:39.383579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 77/167] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:39 2022] GET /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:39.384385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d55c4-1b67-4b8b-9a4b-dc537556e9a6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:39.412942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:39.461469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05114e72-e132-49d6-84fd-7e86ce55b68e None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:39.499765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b463d83-b213-419b-9422-aa25aea239a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:39.500702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b463d83-b213-419b-9422-aa25aea239a4 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:39.500993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b463d83-b213-419b-9422-aa25aea239a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:39.501276 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b463d83-b213-419b-9422-aa25aea239a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:39.513790 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b463d83-b213-419b-9422-aa25aea239a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:39.522190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-0b463d83-b213-419b-9422-aa25aea239a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:39.522190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:39.522190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:39.522190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:39.522190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:39.522190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:39.522190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:39.522190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:39.522190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:39.522190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:39.522190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:39.522190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:39.522190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:39.522190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:39.522190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:39.522190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:39.524010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:39.524010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:39.524010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:39.524010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:39.524010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:39.524010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:39.524010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:39.524010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:39.524010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:39.524010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:39.524010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:39.524010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:39.524010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:39.524010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:39.524010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:39.524010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:39.524786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:39.524786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:39.524786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:39.524786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:39.524786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:39.524786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:39.524786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:39.524786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:39.524786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:39.524786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:39.524786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:39.524786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:39.524786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 91/168] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:39.524786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b463d83-b213-419b-9422-aa25aea239a4 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:39.540987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d55c4-1b67-4b8b-9a4b-dc537556e9a6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:39.589266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35453f-9664-448b-9915-9204390a6cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:39.590236 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35453f-9664-448b-9915-9204390a6cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:39.590616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35453f-9664-448b-9915-9204390a6cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:39.590905 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35453f-9664-448b-9915-9204390a6cf6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:39.603147 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35453f-9664-448b-9915-9204390a6cf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:39.613068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-de35453f-9664-448b-9915-9204390a6cf6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:22:39.613068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:39.613068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:39.613068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:39.613068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:39.613068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:39.613068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:39.613068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:39.613068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:39.613068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:39.613068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:39.613068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:39.613068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:39.613068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:39.613068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:39.613068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:39.615227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:39.615227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:39.615227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:39.615227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:39.615227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:39.615227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:39.615227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:39.615227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:39.615227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:39.615227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:39.615227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:39.615227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:39.615227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:39.615227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:39.615227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:39.615227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:39.617995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:39.617995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:39.617995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:39.617995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:39.617995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:39.617995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:39.617995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:39.617995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:39.617995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:39.617995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:39.617995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:22:39.617995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:22:39.617995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 78/169] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:22:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:39.617995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35453f-9664-448b-9915-9204390a6cf6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:39.642764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b463d83-b213-419b-9422-aa25aea239a4 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:39.644461 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:39.666126 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:39.692883 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca39584-0b1e-4ef4-b55d-07a7365e6c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:39.695185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca39584-0b1e-4ef4-b55d-07a7365e6c5d None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:39.695424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca39584-0b1e-4ef4-b55d-07a7365e6c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:39.695723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca39584-0b1e-4ef4-b55d-07a7365e6c5d None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058/users/fb14ca6f324b4b46b6fbdb67d0fecb95/roles/ff3357541f744b6094e8e5fb5a79aa32` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:39.698404 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca39584-0b1e-4ef4-b55d-07a7365e6c5d None admin] RBAC: Authorizing `identity:create_grant(project_id=53390afb4a404df79ab2d9ae31e8d058, user_id=fb14ca6f324b4b46b6fbdb67d0fecb95, role_id=ff3357541f744b6094e8e5fb5a79aa32)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:39.699946 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca39584-0b1e-4ef4-b55d-07a7365e6c5d None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:39.716667 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 92/170] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:22:39 2022] PUT /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058/users/fb14ca6f324b4b46b6fbdb67d0fecb95/roles/ff3357541f744b6094e8e5fb5a79aa32 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:22:39.717527 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca39584-0b1e-4ef4-b55d-07a7365e6c5d None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:39.767847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35453f-9664-448b-9915-9204390a6cf6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:39.856178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d0e37-85db-4724-843d-99abdb0b3103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:39.856876 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d0e37-85db-4724-843d-99abdb0b3103 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:39.857169 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d0e37-85db-4724-843d-99abdb0b3103 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:39.857456 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d0e37-85db-4724-843d-99abdb0b3103 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:39.858236 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d0e37-85db-4724-843d-99abdb0b3103 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:39.858549 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d0e37-85db-4724-843d-99abdb0b3103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:39.860227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d0e37-85db-4724-843d-99abdb0b3103 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:39.878664 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 79/171] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:39.893607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d0e37-85db-4724-843d-99abdb0b3103 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:39.943376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca39584-0b1e-4ef4-b55d-07a7365e6c5d None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:40.013912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56175e12-d553-4e68-b3cf-a4f49d119b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:40.014601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56175e12-d553-4e68-b3cf-a4f49d119b31 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:40.014892 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56175e12-d553-4e68-b3cf-a4f49d119b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:40.015199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56175e12-d553-4e68-b3cf-a4f49d119b31 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:40.016022 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56175e12-d553-4e68-b3cf-a4f49d119b31 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:40.016336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56175e12-d553-4e68-b3cf-a4f49d119b31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:40.017847 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56175e12-d553-4e68-b3cf-a4f49d119b31 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:40.034261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 93/172] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:22:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:40.040437 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56175e12-d553-4e68-b3cf-a4f49d119b31 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:40.043083 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:40.074685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d0e37-85db-4724-843d-99abdb0b3103 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:40.118185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:40.143253 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639113a-457d-46b2-bb24-e9cf9cb62499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:40.144432 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639113a-457d-46b2-bb24-e9cf9cb62499 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:40.145152 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639113a-457d-46b2-bb24-e9cf9cb62499 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:40.145637 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639113a-457d-46b2-bb24-e9cf9cb62499 None admin] PATH_INFO: `/v3/users/ae81942c6123431c8ac728c83a3710b6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:40.152247 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639113a-457d-46b2-bb24-e9cf9cb62499 None admin] RBAC: Authorizing `identity:get_user(user_id=ae81942c6123431c8ac728c83a3710b6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:40.154092 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639113a-457d-46b2-bb24-e9cf9cb62499 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:40.156401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 80/173] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:39 2022] GET /identity/v3/users/ae81942c6123431c8ac728c83a3710b6 => generated 275 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:40.166137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639113a-457d-46b2-bb24-e9cf9cb62499 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:40.244015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56175e12-d553-4e68-b3cf-a4f49d119b31 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:40.329995 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277d1c8-33ad-4049-8201-613cf13236be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:40.331022 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277d1c8-33ad-4049-8201-613cf13236be None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:40.331386 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277d1c8-33ad-4049-8201-613cf13236be None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:40.331740 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277d1c8-33ad-4049-8201-613cf13236be None admin] PATH_INFO: `/v3/users/ae81942c6123431c8ac728c83a3710b6` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:40.333147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277d1c8-33ad-4049-8201-613cf13236be None admin] RBAC: Authorizing `identity:get_user(user_id=ae81942c6123431c8ac728c83a3710b6)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:40.334819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277d1c8-33ad-4049-8201-613cf13236be None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:40.337284 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 94/174] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:40 2022] GET /identity/v3/users/ae81942c6123431c8ac728c83a3710b6 => generated 275 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:40.342902 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277d1c8-33ad-4049-8201-613cf13236be None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:40.360270 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639113a-457d-46b2-bb24-e9cf9cb62499 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:40.423654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85846cf7-83fc-45a3-b5ea-063e6cdbf035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:40.424329 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85846cf7-83fc-45a3-b5ea-063e6cdbf035 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:40.424623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85846cf7-83fc-45a3-b5ea-063e6cdbf035 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:40.424907 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85846cf7-83fc-45a3-b5ea-063e6cdbf035 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:40.457506 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:40.514757 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85846cf7-83fc-45a3-b5ea-063e6cdbf035 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:40.528581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85846cf7-83fc-45a3-b5ea-063e6cdbf035 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:40.630430 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277d1c8-33ad-4049-8201-613cf13236be None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:40.650568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:40.708306 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 81/175] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 22:22:40.716230 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85846cf7-83fc-45a3-b5ea-063e6cdbf035 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:40.759781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20c35b-0f58-47a0-b944-db9f67cbd6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:40.760465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20c35b-0f58-47a0-b944-db9f67cbd6cf None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:40.760831 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20c35b-0f58-47a0-b944-db9f67cbd6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:40.761111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20c35b-0f58-47a0-b944-db9f67cbd6cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:40.815389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:40.895304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85846cf7-83fc-45a3-b5ea-063e6cdbf035 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:40.959690 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c76a0-4f1d-4df5-a2ec-9d316b48c741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:40.960828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c76a0-4f1d-4df5-a2ec-9d316b48c741 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:40.961118 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c76a0-4f1d-4df5-a2ec-9d316b48c741 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:40.961695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c76a0-4f1d-4df5-a2ec-9d316b48c741 None admin] PATH_INFO: `/v3/projects/ff7c7330c1444db4a0c394169b7517d0` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:40.963293 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c76a0-4f1d-4df5-a2ec-9d316b48c741 None admin] RBAC: Authorizing `identity:get_project(project_id=ff7c7330c1444db4a0c394169b7517d0)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:40.964742 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c76a0-4f1d-4df5-a2ec-9d316b48c741 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:40.966787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 82/176] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:40 2022] GET /identity/v3/projects/ff7c7330c1444db4a0c394169b7517d0 => generated 312 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:40.973873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c76a0-4f1d-4df5-a2ec-9d316b48c741 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:40.976864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20c35b-0f58-47a0-b944-db9f67cbd6cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:40.978706 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20c35b-0f58-47a0-b944-db9f67cbd6cf None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:41.160751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c76a0-4f1d-4df5-a2ec-9d316b48c741 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:41.209189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:41.229142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505d744-10ae-47fe-af11-30f4fb271869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:41.229820 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505d744-10ae-47fe-af11-30f4fb271869 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:41.230110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505d744-10ae-47fe-af11-30f4fb271869 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:41.230393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505d744-10ae-47fe-af11-30f4fb271869 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:41.231384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505d744-10ae-47fe-af11-30f4fb271869 None admin] RBAC: Adding query filter params (role.id=1baba2239751440eb25af0d0fd2d072d, scope.project.id=ff7c7330c1444db4a0c394169b7517d0, user.id=ae81942c6123431c8ac728c83a3710b6) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:41.231701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505d744-10ae-47fe-af11-30f4fb271869 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:41.233145 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505d744-10ae-47fe-af11-30f4fb271869 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:41.245023 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 83/177] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:22:40 2022] GET /identity/v3/role_assignments?user.id=ae81942c6123431c8ac728c83a3710b6&scope.project.id=ff7c7330c1444db4a0c394169b7517d0&role.id=1baba2239751440eb25af0d0fd2d072d => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:41.278756 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 95/178] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 936 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:22:41.284589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505d744-10ae-47fe-af11-30f4fb271869 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:41.415320 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505d744-10ae-47fe-af11-30f4fb271869 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:41.464385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a47a3-2b6d-45d3-8c68-d4f5efe8b18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:41.465301 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a47a3-2b6d-45d3-8c68-d4f5efe8b18a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:41.465593 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a47a3-2b6d-45d3-8c68-d4f5efe8b18a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:41.465881 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a47a3-2b6d-45d3-8c68-d4f5efe8b18a None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:41.467212 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a47a3-2b6d-45d3-8c68-d4f5efe8b18a None admin] RBAC: Authorizing `identity:get_project(project_id=53390afb4a404df79ab2d9ae31e8d058)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:41.468678 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a47a3-2b6d-45d3-8c68-d4f5efe8b18a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:41.470568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 84/179] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:41 2022] GET /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:41.476561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20c35b-0f58-47a0-b944-db9f67cbd6cf None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:41.484916 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:41.592181 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20c35b-0f58-47a0-b944-db9f67cbd6cf None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:41.633150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e2d407-85b6-4c14-9d2a-e8d58a7c54df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:41.633885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2d407-85b6-4c14-9d2a-e8d58a7c54df None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:41.634178 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2d407-85b6-4c14-9d2a-e8d58a7c54df None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:41.634463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2d407-85b6-4c14-9d2a-e8d58a7c54df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:41.635345 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e2d407-85b6-4c14-9d2a-e8d58a7c54df None admin] RBAC: Adding query filter params (role.id=61397d4a1ccd4a8ca479f98186ed489b, scope.project.id=53390afb4a404df79ab2d9ae31e8d058, user.id=ae81942c6123431c8ac728c83a3710b6) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:41.635664 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e2d407-85b6-4c14-9d2a-e8d58a7c54df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:41.637130 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e2d407-85b6-4c14-9d2a-e8d58a7c54df None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:41.648669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 96/180] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:22:41 2022] GET /identity/v3/role_assignments?user.id=ae81942c6123431c8ac728c83a3710b6&scope.project.id=53390afb4a404df79ab2d9ae31e8d058&role.id=61397d4a1ccd4a8ca479f98186ed489b => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:41.906337 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdc07f-8d6b-41ac-93fc-fe08724ae033 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:41.906715 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdc07f-8d6b-41ac-93fc-fe08724ae033 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:41.906912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdc07f-8d6b-41ac-93fc-fe08724ae033 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:41.908673 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 85/181] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:41.916475 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09267dc4-f5d5-479b-903b-3ab938620706 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:41.916475 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09267dc4-f5d5-479b-903b-3ab938620706 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:41.916475 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09267dc4-f5d5-479b-903b-3ab938620706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:41.934105 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-09267dc4-f5d5-479b-903b-3ab938620706 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:41.935387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-09267dc4-f5d5-479b-903b-3ab938620706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:42.015627 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:42.020123 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 97/182] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:42.044597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdc07f-8d6b-41ac-93fc-fe08724ae033 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:42.107480 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:42.157634 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdc07f-8d6b-41ac-93fc-fe08724ae033 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:42.194142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881bc8f-8090-4162-9b38-63cdc54c0f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:42.195487 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881bc8f-8090-4162-9b38-63cdc54c0f85 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:42.195590 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881bc8f-8090-4162-9b38-63cdc54c0f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:42.195882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881bc8f-8090-4162-9b38-63cdc54c0f85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:42.206459 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7881bc8f-8090-4162-9b38-63cdc54c0f85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:42.215070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c8411-3fd8-4063-bf1c-f30b22193849 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:42.215226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7881bc8f-8090-4162-9b38-63cdc54c0f85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:22:42.215226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:42.215226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:42.215226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:42.215226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:42.215226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:42.215226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:42.215226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:42.215226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:42.215226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:42.215226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:42.215226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:42.215226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:42.215226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:42.215226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:42.215226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:42.216067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:42.216067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:42.216067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:42.216067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:42.216067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:42.216067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:42.216067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:42.216067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:42.216067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:42.216067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:42.216067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:42.216067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:42.216067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:42.216067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:42.216067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:42.216067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:42.216851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:42.216851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:42.216851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:42.216851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:42.216851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:42.216851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:42.216851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:42.216851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:42.216851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:42.216851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:42.216851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:22:42.216851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:22:42.217435 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c8411-3fd8-4063-bf1c-f30b22193849 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:42.217435 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c8411-3fd8-4063-bf1c-f30b22193849 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:42.217559 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 86/183] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:22:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:42.217625 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 98/184] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:42.222006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881bc8f-8090-4162-9b38-63cdc54c0f85 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:42.223531 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bbf61-ba1c-409f-98eb-577576c638c9 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:42.223826 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bbf61-ba1c-409f-98eb-577576c638c9 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:42.224115 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bbf61-ba1c-409f-98eb-577576c638c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:42.243013 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-679bbf61-ba1c-409f-98eb-577576c638c9 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:42.244269 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-679bbf61-ba1c-409f-98eb-577576c638c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:42.324500 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 99/185] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:22:42.350787 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bbf61-ba1c-409f-98eb-577576c638c9 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:42.352408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881bc8f-8090-4162-9b38-63cdc54c0f85 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:42.373217 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:42.401594 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee55ac-bd46-4e0d-b60d-b58ed826a5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:42.402559 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee55ac-bd46-4e0d-b60d-b58ed826a5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:42.403053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee55ac-bd46-4e0d-b60d-b58ed826a5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:42.404097 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee55ac-bd46-4e0d-b60d-b58ed826a5f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:42.405077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee55ac-bd46-4e0d-b60d-b58ed826a5f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:42.405568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee55ac-bd46-4e0d-b60d-b58ed826a5f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:42.407156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee55ac-bd46-4e0d-b60d-b58ed826a5f8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:42.419357 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 87/186] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:42.425014 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee55ac-bd46-4e0d-b60d-b58ed826a5f8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:42.477445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bbf61-ba1c-409f-98eb-577576c638c9 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:42.509474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:42.516470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bc824-ff34-4a31-b357-9902ab48248f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:42.517278 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bc824-ff34-4a31-b357-9902ab48248f None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:42.517577 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bc824-ff34-4a31-b357-9902ab48248f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:42.517853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bc824-ff34-4a31-b357-9902ab48248f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:42.519495 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bc824-ff34-4a31-b357-9902ab48248f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:42.520867 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bc824-ff34-4a31-b357-9902ab48248f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:42.522852 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 100/187] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:22:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:42.527658 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b337f-a28d-4ce6-bd6d-b91344e7f324 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:42.527957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b337f-a28d-4ce6-bd6d-b91344e7f324 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:42.528246 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b337f-a28d-4ce6-bd6d-b91344e7f324 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:42.529683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 101/188] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:42.530220 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b337f-a28d-4ce6-bd6d-b91344e7f324 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:42.542842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee55ac-bd46-4e0d-b60d-b58ed826a5f8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:42.585143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12130972-7cca-4a0b-82b4-916e69a66b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:42.586836 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12130972-7cca-4a0b-82b4-916e69a66b56 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:42.587670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12130972-7cca-4a0b-82b4-916e69a66b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:42.588153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12130972-7cca-4a0b-82b4-916e69a66b56 None admin] PATH_INFO: `/v3/users/ad2e3cebb6354f6aa4d0c19f24807e4e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:42.605723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12130972-7cca-4a0b-82b4-916e69a66b56 None admin] RBAC: Authorizing `identity:get_user(user_id=ad2e3cebb6354f6aa4d0c19f24807e4e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:42.607227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12130972-7cca-4a0b-82b4-916e69a66b56 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:42.609087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 88/189] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:42 2022] GET /identity/v3/users/ad2e3cebb6354f6aa4d0c19f24807e4e => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:42.609087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7132d25-4bf2-4076-98ad-c2b657f2cf97 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:42.609087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7132d25-4bf2-4076-98ad-c2b657f2cf97 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:42.609087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7132d25-4bf2-4076-98ad-c2b657f2cf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:42.628870 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e7132d25-4bf2-4076-98ad-c2b657f2cf97 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:42.630069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e7132d25-4bf2-4076-98ad-c2b657f2cf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:42.645171 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b337f-a28d-4ce6-bd6d-b91344e7f324 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:42.658184 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:42.682299 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee55929-321e-420a-885f-3dd80afb9b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:42.682912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee55929-321e-420a-885f-3dd80afb9b1c None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:42.683216 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee55929-321e-420a-885f-3dd80afb9b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:42.683500 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee55929-321e-420a-885f-3dd80afb9b1c None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:42.684216 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee55929-321e-420a-885f-3dd80afb9b1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:42.685656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee55929-321e-420a-885f-3dd80afb9b1c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:42.705263 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 89/190] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:42.706237 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7132d25-4bf2-4076-98ad-c2b657f2cf97 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:42.717341 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 102/191] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 22:22:42 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:22:42.733533 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee55929-321e-420a-885f-3dd80afb9b1c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:42.803906 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:42.840095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7132d25-4bf2-4076-98ad-c2b657f2cf97 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:42.862074 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee55929-321e-420a-885f-3dd80afb9b1c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:42.884330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834be145-f1c5-4dda-9a1a-7dc9489bdcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:42.885210 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834be145-f1c5-4dda-9a1a-7dc9489bdcf8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:42.885353 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834be145-f1c5-4dda-9a1a-7dc9489bdcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:42.885532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834be145-f1c5-4dda-9a1a-7dc9489bdcf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:42.909680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ba4e96-1cbc-4370-bf60-e5eaadf905c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:42.910537 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba4e96-1cbc-4370-bf60-e5eaadf905c6 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:42.910825 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba4e96-1cbc-4370-bf60-e5eaadf905c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:42.911181 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ba4e96-1cbc-4370-bf60-e5eaadf905c6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:42.922804 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ba4e96-1cbc-4370-bf60-e5eaadf905c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:42.928246 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:42.931596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-97ba4e96-1cbc-4370-bf60-e5eaadf905c6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:22:42.931596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:42.931596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:42.931596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:42.931596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:42.931596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:42.931596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:42.931596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:42.931596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:42.931596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:42.931596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:42.931596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:42.931596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:42.931596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:42.931596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:42.931596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:42.932526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:42.932526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:42.932526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:42.932526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:42.932526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:42.932526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:42.932526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:42.932526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:42.932526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:42.932526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:42.932526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:42.932526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:42.932526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:42.932526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:42.932526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:42.932526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:42.933319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:42.933319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:42.933319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:42.933319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:42.933319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:42.933319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:42.933319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:42.933319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:42.933319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:42.933319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:42.933319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:22:42.933319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:42.933319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 103/192] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:22:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:22:42.934095 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8979749-3cbd-4002-a13f-b80ff4abb646 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:42.934388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8979749-3cbd-4002-a13f-b80ff4abb646 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:42.934678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8979749-3cbd-4002-a13f-b80ff4abb646 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:42.936126 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 104/193] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:42.939979 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8979749-3cbd-4002-a13f-b80ff4abb646 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:42.985937 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834be145-f1c5-4dda-9a1a-7dc9489bdcf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:42.987972 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834be145-f1c5-4dda-9a1a-7dc9489bdcf8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:43.078041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8979749-3cbd-4002-a13f-b80ff4abb646 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:43.096544 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:43.115447 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 90/194] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:22:43.117716 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf3ac3-9fac-4982-b17c-ddb9d4904b24 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:43.118644 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf3ac3-9fac-4982-b17c-ddb9d4904b24 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:43.119306 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf3ac3-9fac-4982-b17c-ddb9d4904b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:43.135134 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc91f2-02ab-4ea2-8728-c4d43a9dbe49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:43.136334 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc91f2-02ab-4ea2-8728-c4d43a9dbe49 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:43.136776 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc91f2-02ab-4ea2-8728-c4d43a9dbe49 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:43.137158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc91f2-02ab-4ea2-8728-c4d43a9dbe49 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:43.138053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc91f2-02ab-4ea2-8728-c4d43a9dbe49 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:43.138474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc91f2-02ab-4ea2-8728-c4d43a9dbe49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:43.140005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc91f2-02ab-4ea2-8728-c4d43a9dbe49 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:43.140782 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a2bf3ac3-9fac-4982-b17c-ddb9d4904b24 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:43.142419 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a2bf3ac3-9fac-4982-b17c-ddb9d4904b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:43.154159 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 105/195] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:22:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:43.155483 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc91f2-02ab-4ea2-8728-c4d43a9dbe49 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:43.209602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:43.220619 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 91/196] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:43.226951 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf3ac3-9fac-4982-b17c-ddb9d4904b24 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:43.273455 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc91f2-02ab-4ea2-8728-c4d43a9dbe49 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:43.310290 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1cf97-9287-4721-989b-c1bd15647fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:43.311247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1cf97-9287-4721-989b-c1bd15647fdb None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:43.311543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1cf97-9287-4721-989b-c1bd15647fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:43.311829 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1cf97-9287-4721-989b-c1bd15647fdb None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:43.313110 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1cf97-9287-4721-989b-c1bd15647fdb None admin] RBAC: Authorizing `identity:get_project(project_id=6525e5222e3342a3a26a6369518c7577)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:43.314487 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1cf97-9287-4721-989b-c1bd15647fdb None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:43.316388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 106/197] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:43 2022] GET /identity/v3/projects/6525e5222e3342a3a26a6369518c7577 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:43.317058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1cf97-9287-4721-989b-c1bd15647fdb None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:43.353916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf3ac3-9fac-4982-b17c-ddb9d4904b24 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:43.386487 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:43.393864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6de035-6b1b-4bcd-bd03-b1faa4764d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:43.395226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6de035-6b1b-4bcd-bd03-b1faa4764d84 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:43.395830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6de035-6b1b-4bcd-bd03-b1faa4764d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:43.396316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6de035-6b1b-4bcd-bd03-b1faa4764d84 None admin] PATH_INFO: `/v3/users/ad2e3cebb6354f6aa4d0c19f24807e4e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:43.398078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6de035-6b1b-4bcd-bd03-b1faa4764d84 None admin] RBAC: Authorizing `identity:get_user(user_id=ad2e3cebb6354f6aa4d0c19f24807e4e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:43.399823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6de035-6b1b-4bcd-bd03-b1faa4764d84 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:43.401746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 92/198] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:43 2022] GET /identity/v3/users/ad2e3cebb6354f6aa4d0c19f24807e4e => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:43.402724 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6de035-6b1b-4bcd-bd03-b1faa4764d84 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:43.449006 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1cf97-9287-4721-989b-c1bd15647fdb None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:43.467292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:43.487348 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d509c2-2674-43c9-9948-6afd26e721dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:43.488295 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d509c2-2674-43c9-9948-6afd26e721dc None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:43.488640 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d509c2-2674-43c9-9948-6afd26e721dc None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:43.488923 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d509c2-2674-43c9-9948-6afd26e721dc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:43.501038 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d509c2-2674-43c9-9948-6afd26e721dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:43.511521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-47d509c2-2674-43c9-9948-6afd26e721dc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:22:43.511521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:43.511521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:43.511521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:43.511521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:43.511521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:43.511521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:43.511521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:43.511521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:43.511521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:43.511521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:43.511521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:43.511521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:43.511521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:43.511521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:43.511521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:43.512445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:43.512445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:43.512445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:43.512445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:43.512445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:43.512445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:43.512445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:43.512445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:43.512445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:43.512445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:43.512445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:43.512445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:43.512445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:43.512445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:43.512445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:43.512445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:43.513251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:43.513251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:43.513251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:43.513251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:43.513251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:43.513251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:43.513251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:43.513251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:43.513251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:43.513251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:43.513251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:22:43.513251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:43.513251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 107/199] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:43.513898 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d509c2-2674-43c9-9948-6afd26e721dc None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:43.519773 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6de035-6b1b-4bcd-bd03-b1faa4764d84 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:43.556146 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed56b1-872f-4956-9bc9-0474b1d813e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:43.557164 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed56b1-872f-4956-9bc9-0474b1d813e2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:43.557658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed56b1-872f-4956-9bc9-0474b1d813e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:43.558145 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed56b1-872f-4956-9bc9-0474b1d813e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:43.559260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed56b1-872f-4956-9bc9-0474b1d813e2 None admin] RBAC: Adding query filter params (role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.project.id=6525e5222e3342a3a26a6369518c7577, user.id=ad2e3cebb6354f6aa4d0c19f24807e4e) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:43.559816 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed56b1-872f-4956-9bc9-0474b1d813e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:43.561491 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed56b1-872f-4956-9bc9-0474b1d813e2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:43.567924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 93/200] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:22:43 2022] GET /identity/v3/role_assignments?user.id=ad2e3cebb6354f6aa4d0c19f24807e4e&scope.project.id=6525e5222e3342a3a26a6369518c7577&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:43.569012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed56b1-872f-4956-9bc9-0474b1d813e2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:43.633483 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d509c2-2674-43c9-9948-6afd26e721dc None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:43.636091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:43.675915 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d32fc-9ffc-4a19-b996-7d41a8a30173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:43.676641 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d32fc-9ffc-4a19-b996-7d41a8a30173 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:43.676925 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d32fc-9ffc-4a19-b996-7d41a8a30173 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:43.677203 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d32fc-9ffc-4a19-b996-7d41a8a30173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:43.684514 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed56b1-872f-4956-9bc9-0474b1d813e2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:43.714244 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:43.733667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf6113-7b85-4888-9b08-62c535169123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:43.734733 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf6113-7b85-4888-9b08-62c535169123 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:43.735388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf6113-7b85-4888-9b08-62c535169123 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:43.735875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf6113-7b85-4888-9b08-62c535169123 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:43.736948 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf6113-7b85-4888-9b08-62c535169123 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:43.737493 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf6113-7b85-4888-9b08-62c535169123 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:43.739151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf6113-7b85-4888-9b08-62c535169123 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:43.749774 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 94/201] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:43.755816 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf6113-7b85-4888-9b08-62c535169123 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:43.791492 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d32fc-9ffc-4a19-b996-7d41a8a30173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:43.793377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d32fc-9ffc-4a19-b996-7d41a8a30173 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:43.869437 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf6113-7b85-4888-9b08-62c535169123 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:43.902187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 108/202] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:22:43.907757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d32fc-9ffc-4a19-b996-7d41a8a30173 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:43.909111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012a845-fa34-43c6-bdea-1fbb2295e7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:43.910636 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:43.911611 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012a845-fa34-43c6-bdea-1fbb2295e7fc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:43.912200 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012a845-fa34-43c6-bdea-1fbb2295e7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:43.912755 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012a845-fa34-43c6-bdea-1fbb2295e7fc None admin] PATH_INFO: `/v3/users/fb14ca6f324b4b46b6fbdb67d0fecb95` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:43.914727 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012a845-fa34-43c6-bdea-1fbb2295e7fc None admin] RBAC: Authorizing `identity:get_user(user_id=fb14ca6f324b4b46b6fbdb67d0fecb95)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:43.917092 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012a845-fa34-43c6-bdea-1fbb2295e7fc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:43.919425 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 95/203] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:43 2022] GET /identity/v3/users/fb14ca6f324b4b46b6fbdb67d0fecb95 => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:43.925076 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012a845-fa34-43c6-bdea-1fbb2295e7fc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:43.986617 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:44.016353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d32fc-9ffc-4a19-b996-7d41a8a30173 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:44.044057 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012a845-fa34-43c6-bdea-1fbb2295e7fc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:44.072680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b77c74-0868-477f-82e1-b1d3ad0a3a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:44.073565 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b77c74-0868-477f-82e1-b1d3ad0a3a83 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:44.073880 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b77c74-0868-477f-82e1-b1d3ad0a3a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:44.074411 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b77c74-0868-477f-82e1-b1d3ad0a3a83 None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:44.075530 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b77c74-0868-477f-82e1-b1d3ad0a3a83 None admin] RBAC: Authorizing `identity:get_project(project_id=6525e5222e3342a3a26a6369518c7577)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:44.076993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b77c74-0868-477f-82e1-b1d3ad0a3a83 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:44.079071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 109/204] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:43 2022] GET /identity/v3/projects/6525e5222e3342a3a26a6369518c7577 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:44.083230 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65218c1-f527-403c-9848-c7119adda520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:44.084138 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65218c1-f527-403c-9848-c7119adda520 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:44.084570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b77c74-0868-477f-82e1-b1d3ad0a3a83 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:44.084855 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65218c1-f527-403c-9848-c7119adda520 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:44.085884 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65218c1-f527-403c-9848-c7119adda520 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:44.170361 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65218c1-f527-403c-9848-c7119adda520 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:44.172238 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65218c1-f527-403c-9848-c7119adda520 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:44.206764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:44.233615 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:44.247971 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b77c74-0868-477f-82e1-b1d3ad0a3a83 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:44.285968 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421ca08-0494-4e54-9992-23e5408a121d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:44.286632 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421ca08-0494-4e54-9992-23e5408a121d None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:44.286916 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421ca08-0494-4e54-9992-23e5408a121d None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:44.287236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421ca08-0494-4e54-9992-23e5408a121d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:44.288648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421ca08-0494-4e54-9992-23e5408a121d None admin] RBAC: Adding query filter params (role.id=61397d4a1ccd4a8ca479f98186ed489b, scope.project.id=6525e5222e3342a3a26a6369518c7577, user.id=ad2e3cebb6354f6aa4d0c19f24807e4e) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:44.288892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 96/205] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:22:44.289214 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421ca08-0494-4e54-9992-23e5408a121d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:44.292592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421ca08-0494-4e54-9992-23e5408a121d None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:44.295346 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65218c1-f527-403c-9848-c7119adda520 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:44.296642 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 110/206] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:22:44 2022] GET /identity/v3/role_assignments?user.id=ad2e3cebb6354f6aa4d0c19f24807e4e&scope.project.id=6525e5222e3342a3a26a6369518c7577&role.id=61397d4a1ccd4a8ca479f98186ed489b => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:44.406958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65218c1-f527-403c-9848-c7119adda520 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:44.445264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9e7d8-e29a-486c-842a-8d62792f3e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:44.446512 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9e7d8-e29a-486c-842a-8d62792f3e1b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:44.447038 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9e7d8-e29a-486c-842a-8d62792f3e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:44.447521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9e7d8-e29a-486c-842a-8d62792f3e1b None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:44.449017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9e7d8-e29a-486c-842a-8d62792f3e1b None admin] RBAC: Authorizing `identity:get_project(project_id=53390afb4a404df79ab2d9ae31e8d058)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:44.450676 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9e7d8-e29a-486c-842a-8d62792f3e1b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:44.452676 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 97/207] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:44 2022] GET /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:44.459160 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421ca08-0494-4e54-9992-23e5408a121d None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:44.577231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421ca08-0494-4e54-9992-23e5408a121d None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:44.623758 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c9a18-f8f6-4b32-9b2d-5b6f9f98c687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:44.624471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c9a18-f8f6-4b32-9b2d-5b6f9f98c687 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:44.624759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c9a18-f8f6-4b32-9b2d-5b6f9f98c687 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:44.625038 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c9a18-f8f6-4b32-9b2d-5b6f9f98c687 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:44.625872 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c9a18-f8f6-4b32-9b2d-5b6f9f98c687 None admin] RBAC: Adding query filter params (role.id=ff3357541f744b6094e8e5fb5a79aa32, scope.project.id=53390afb4a404df79ab2d9ae31e8d058, user.id=fb14ca6f324b4b46b6fbdb67d0fecb95) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:44.626240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c9a18-f8f6-4b32-9b2d-5b6f9f98c687 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:44.627710 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c9a18-f8f6-4b32-9b2d-5b6f9f98c687 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:44.636241 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:44.640131 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 111/208] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:22:44 2022] GET /identity/v3/role_assignments?user.id=fb14ca6f324b4b46b6fbdb67d0fecb95&scope.project.id=53390afb4a404df79ab2d9ae31e8d058&role.id=ff3357541f744b6094e8e5fb5a79aa32 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:45.113633 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53897d17-001f-4332-a13e-cd5fa4d4854e None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:45.114030 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53897d17-001f-4332-a13e-cd5fa4d4854e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:45.114208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53897d17-001f-4332-a13e-cd5fa4d4854e None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:45.115765 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 98/209] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:45.125799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a44f5-cc86-4d89-af11-26bd6660a1fd None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:45.125799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a44f5-cc86-4d89-af11-26bd6660a1fd None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:45.125799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a44f5-cc86-4d89-af11-26bd6660a1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:45.143884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-828a44f5-cc86-4d89-af11-26bd6660a1fd None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:45.145082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-828a44f5-cc86-4d89-af11-26bd6660a1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:45.229417 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 112/210] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:45.256806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53897d17-001f-4332-a13e-cd5fa4d4854e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:45.290191 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:45.339938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6618e7b-df5a-4042-a7b0-dc55b5ebec04 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:45.340124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6618e7b-df5a-4042-a7b0-dc55b5ebec04 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:45.340418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6618e7b-df5a-4042-a7b0-dc55b5ebec04 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:45.342050 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 113/211] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:45.346937 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d5917-0323-496a-873f-e511fb5b62b9 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:45.347137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d5917-0323-496a-873f-e511fb5b62b9 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:45.347366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d5917-0323-496a-873f-e511fb5b62b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:45.369391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-9b4d5917-0323-496a-873f-e511fb5b62b9 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:45.370520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-9b4d5917-0323-496a-873f-e511fb5b62b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:45.379702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53897d17-001f-4332-a13e-cd5fa4d4854e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:45.421243 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e29da5-cdac-4d4d-88d6-c64b4c3b4ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:45.422538 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e29da5-cdac-4d4d-88d6-c64b4c3b4ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:45.423114 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e29da5-cdac-4d4d-88d6-c64b4c3b4ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:45.423587 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e29da5-cdac-4d4d-88d6-c64b4c3b4ff1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:45.436025 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e29da5-cdac-4d4d-88d6-c64b4c3b4ff1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:45.443837 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 114/212] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:22:45.446313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a4e29da5-cdac-4d4d-88d6-c64b4c3b4ff1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:45.446313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:45.446313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:45.446313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:45.446313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:45.446313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:45.446313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:45.446313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:45.446313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:45.446313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:45.446313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:45.446313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:45.446313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:45.446313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:45.446313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:45.446313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:45.447277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:45.447277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:45.447277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:45.447277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:45.447277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:45.447277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:45.447277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:45.447277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:45.447277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:45.447277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:45.447277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:45.447277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:45.447277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:45.447277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:45.447277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:45.447277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:45.448470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:45.448470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:45.448470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:45.448470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:45.448470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:45.448470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:45.448470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:45.448470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:45.448470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:45.448470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:45.448470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:45.448470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:22:45.448470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 99/213] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:45.458960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d5917-0323-496a-873f-e511fb5b62b9 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:45.467944 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e29da5-cdac-4d4d-88d6-c64b4c3b4ff1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:45.529804 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:45.561938 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:45.580553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e29da5-cdac-4d4d-88d6-c64b4c3b4ff1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:45.586811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d5917-0323-496a-873f-e511fb5b62b9 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:45.619730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f16a7-56ba-4045-b552-ed72b007a308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:45.620786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f16a7-56ba-4045-b552-ed72b007a308 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:45.621243 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f16a7-56ba-4045-b552-ed72b007a308 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:45.621690 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f16a7-56ba-4045-b552-ed72b007a308 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:45.623054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f16a7-56ba-4045-b552-ed72b007a308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:45.624704 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f16a7-56ba-4045-b552-ed72b007a308 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:45.626546 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 100/214] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:22:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:45.632115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f16a7-56ba-4045-b552-ed72b007a308 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:45.635406 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdf2c5-81ed-40f5-a197-03780ae2160f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:45.636367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdf2c5-81ed-40f5-a197-03780ae2160f None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:45.636893 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdf2c5-81ed-40f5-a197-03780ae2160f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:45.637378 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdf2c5-81ed-40f5-a197-03780ae2160f None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:45.638322 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cdf2c5-81ed-40f5-a197-03780ae2160f None admin] RBAC: Adding query filter params (name=member) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:45.638822 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cdf2c5-81ed-40f5-a197-03780ae2160f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:45.640416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cdf2c5-81ed-40f5-a197-03780ae2160f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:45.654838 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 115/215] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:45.656012 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdf2c5-81ed-40f5-a197-03780ae2160f None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:45.755017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f16a7-56ba-4045-b552-ed72b007a308 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:45.763060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdf2c5-81ed-40f5-a197-03780ae2160f None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:45.792116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:45.795771 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ce8ce-795c-4415-acb5-574b3a1f2f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:45.796532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ce8ce-795c-4415-acb5-574b3a1f2f89 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:45.797222 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ce8ce-795c-4415-acb5-574b3a1f2f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:45.797585 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ce8ce-795c-4415-acb5-574b3a1f2f89 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:45.798844 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ce8ce-795c-4415-acb5-574b3a1f2f89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:45.800418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ce8ce-795c-4415-acb5-574b3a1f2f89 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:45.808586 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee087df8-8a91-4c63-abf1-10c4ec6c51db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:45.810350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee087df8-8a91-4c63-abf1-10c4ec6c51db None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:45.811090 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee087df8-8a91-4c63-abf1-10c4ec6c51db None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:45.811804 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee087df8-8a91-4c63-abf1-10c4ec6c51db None admin] PATH_INFO: `/v3/users/7faa873fa2c34876a03eed73038a6dd4` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:45.835966 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee087df8-8a91-4c63-abf1-10c4ec6c51db None admin] RBAC: Authorizing `identity:get_user(user_id=7faa873fa2c34876a03eed73038a6dd4)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:45.836772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 101/216] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 22:22:45 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:22:45.839485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee087df8-8a91-4c63-abf1-10c4ec6c51db None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:45.842680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 116/217] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:45 2022] GET /identity/v3/users/7faa873fa2c34876a03eed73038a6dd4 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:45.853065 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ce8ce-795c-4415-acb5-574b3a1f2f89 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:45.874125 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:45.987263 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ce8ce-795c-4415-acb5-574b3a1f2f89 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:46.027066 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34d7cd-c431-4bb6-9dd9-59e4875e6965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:46.028075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34d7cd-c431-4bb6-9dd9-59e4875e6965 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:46.028568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34d7cd-c431-4bb6-9dd9-59e4875e6965 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:46.029045 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34d7cd-c431-4bb6-9dd9-59e4875e6965 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:46.104425 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34d7cd-c431-4bb6-9dd9-59e4875e6965 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:46.105910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa34d7cd-c431-4bb6-9dd9-59e4875e6965 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:46.117918 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:46.165715 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5cdc59-100e-4962-a1c0-35e977bd5b0a None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:46.166033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5cdc59-100e-4962-a1c0-35e977bd5b0a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:46.166296 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5cdc59-100e-4962-a1c0-35e977bd5b0a None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:46.167820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 117/218] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:46.173868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b8295-0894-48df-817a-4464936a1e6a None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:46.174010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b8295-0894-48df-817a-4464936a1e6a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:46.174195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b8295-0894-48df-817a-4464936a1e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:46.198956 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-243b8295-0894-48df-817a-4464936a1e6a None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:46.200342 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-243b8295-0894-48df-817a-4464936a1e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:46.245126 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 102/219] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:22:46.250671 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34d7cd-c431-4bb6-9dd9-59e4875e6965 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:46.298898 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 118/220] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:46.322192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b8295-0894-48df-817a-4464936a1e6a None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:46.367486 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34d7cd-c431-4bb6-9dd9-59e4875e6965 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:46.377965 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:46.421609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7005a5-ab1c-4dc8-8702-89ce71d171cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:46.422489 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7005a5-ab1c-4dc8-8702-89ce71d171cb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:46.422781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7005a5-ab1c-4dc8-8702-89ce71d171cb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:46.423070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7005a5-ab1c-4dc8-8702-89ce71d171cb None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:46.424376 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7005a5-ab1c-4dc8-8702-89ce71d171cb None admin] RBAC: Authorizing `identity:get_project(project_id=6525e5222e3342a3a26a6369518c7577)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:46.425828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7005a5-ab1c-4dc8-8702-89ce71d171cb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:46.427911 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 103/221] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:46 2022] GET /identity/v3/projects/6525e5222e3342a3a26a6369518c7577 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:46.433472 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7005a5-ab1c-4dc8-8702-89ce71d171cb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:46.457570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b8295-0894-48df-817a-4464936a1e6a None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:46.476122 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:46.523708 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7804911-563e-4e90-9ddd-471ffec0e9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:46.524535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7804911-563e-4e90-9ddd-471ffec0e9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:46.524823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7804911-563e-4e90-9ddd-471ffec0e9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:46.525102 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7804911-563e-4e90-9ddd-471ffec0e9b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:46.548192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7804911-563e-4e90-9ddd-471ffec0e9b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:46.548793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7005a5-ab1c-4dc8-8702-89ce71d171cb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:46.559511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-e7804911-563e-4e90-9ddd-471ffec0e9b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:22:46.559511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:46.559511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:46.559511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:46.559511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:46.559511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:46.559511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:46.559511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:46.559511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:46.559511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:46.559511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:46.559511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:46.559511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:46.559511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:46.559511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:46.559511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:46.560447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:46.560447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:46.560447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:46.560447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:46.560447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:46.560447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:46.560447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:46.560447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:46.560447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:46.560447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:46.560447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:46.560447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:46.560447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:46.560447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:46.560447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:46.560447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:46.561237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:46.561237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:46.561237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:46.561237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:46.561237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:46.561237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:46.561237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:46.561237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:46.561237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:46.561237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:46.561237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:22:46.561237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:46.568340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 119/222] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:22:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:22:46.569603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7804911-563e-4e90-9ddd-471ffec0e9b5 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:46.588285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78734ae3-b7f1-4632-9fbe-af8006076f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:46.588943 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78734ae3-b7f1-4632-9fbe-af8006076f67 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:46.589228 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78734ae3-b7f1-4632-9fbe-af8006076f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:46.589503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78734ae3-b7f1-4632-9fbe-af8006076f67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:46.590343 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78734ae3-b7f1-4632-9fbe-af8006076f67 None admin] RBAC: Adding query filter params (role.id=1baba2239751440eb25af0d0fd2d072d, scope.project.id=6525e5222e3342a3a26a6369518c7577, user.id=7faa873fa2c34876a03eed73038a6dd4) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:46.590654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78734ae3-b7f1-4632-9fbe-af8006076f67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:46.598770 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78734ae3-b7f1-4632-9fbe-af8006076f67 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:46.598770 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 104/223] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:22:46 2022] GET /identity/v3/role_assignments?user.id=7faa873fa2c34876a03eed73038a6dd4&scope.project.id=6525e5222e3342a3a26a6369518c7577&role.id=1baba2239751440eb25af0d0fd2d072d => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:46.756055 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95374cbb-5ee2-4875-8251-725e9d0f0c61 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:46.756427 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95374cbb-5ee2-4875-8251-725e9d0f0c61 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:46.756624 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95374cbb-5ee2-4875-8251-725e9d0f0c61 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:46.758655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 105/224] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:22:46.759604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7804911-563e-4e90-9ddd-471ffec0e9b5 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:46.768545 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3580a4-55f8-4ae0-8e7d-c063aca2fd69 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:46.769130 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3580a4-55f8-4ae0-8e7d-c063aca2fd69 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:46.769608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3580a4-55f8-4ae0-8e7d-c063aca2fd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:46.791827 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ce3580a4-55f8-4ae0-8e7d-c063aca2fd69 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:46.793344 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ce3580a4-55f8-4ae0-8e7d-c063aca2fd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:46.829053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c7c25-ea71-4d94-bf7f-3455ff6ee755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:46.830111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c7c25-ea71-4d94-bf7f-3455ff6ee755 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:46.830594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c7c25-ea71-4d94-bf7f-3455ff6ee755 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:46.831116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c7c25-ea71-4d94-bf7f-3455ff6ee755 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:46.832089 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c7c25-ea71-4d94-bf7f-3455ff6ee755 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:46.832757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c7c25-ea71-4d94-bf7f-3455ff6ee755 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:46.836653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c7c25-ea71-4d94-bf7f-3455ff6ee755 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:46.844503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:46.857545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 120/225] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:46.863511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c7c25-ea71-4d94-bf7f-3455ff6ee755 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:46.910219 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:46.922164 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 106/226] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:22:46.951973 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3580a4-55f8-4ae0-8e7d-c063aca2fd69 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:47.040190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c7c25-ea71-4d94-bf7f-3455ff6ee755 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:47.118453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdf8a5-ea9c-44a7-bfb2-7b770e2f7104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:47.119745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdf8a5-ea9c-44a7-bfb2-7b770e2f7104 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:47.120233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdf8a5-ea9c-44a7-bfb2-7b770e2f7104 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:47.120712 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdf8a5-ea9c-44a7-bfb2-7b770e2f7104 None admin] PATH_INFO: `/v3/users/ad2e3cebb6354f6aa4d0c19f24807e4e` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:47.122255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdf8a5-ea9c-44a7-bfb2-7b770e2f7104 None admin] RBAC: Authorizing `identity:get_user(user_id=ad2e3cebb6354f6aa4d0c19f24807e4e)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:47.123957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdf8a5-ea9c-44a7-bfb2-7b770e2f7104 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:47.127655 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 121/227] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:46 2022] GET /identity/v3/users/ad2e3cebb6354f6aa4d0c19f24807e4e => generated 283 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:47.136249 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdf8a5-ea9c-44a7-bfb2-7b770e2f7104 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:47.144624 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3580a4-55f8-4ae0-8e7d-c063aca2fd69 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:47.207642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e212d-1aae-4b90-aeed-9c26f87889f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:47.208466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e212d-1aae-4b90-aeed-9c26f87889f3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:47.208753 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e212d-1aae-4b90-aeed-9c26f87889f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:47.209038 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e212d-1aae-4b90-aeed-9c26f87889f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:47.229151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4e212d-1aae-4b90-aeed-9c26f87889f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:47.243731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4e4e212d-1aae-4b90-aeed-9c26f87889f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:22:47.243731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:47.243731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:47.243731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:47.243731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:47.243731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:47.243731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:47.243731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:47.243731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:47.243731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:47.243731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:47.243731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:47.243731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:47.243731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:47.243731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:47.243731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:47.244604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:47.244604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:47.244604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:47.244604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:47.244604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:47.244604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:47.244604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:47.244604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:47.244604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:47.244604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:47.244604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:47.244604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:47.244604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:47.244604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:47.244604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:47.244604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:47.245384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:47.245384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:47.245384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:47.245384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:47.245384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:47.245384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:47.245384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:47.245384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:47.245384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:47.245384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:47.245384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:22:47.245384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:22:47.245384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 107/228] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:22:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:47.251905 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e212d-1aae-4b90-aeed-9c26f87889f3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:47.306263 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:47.310234 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdf8a5-ea9c-44a7-bfb2-7b770e2f7104 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:47.320813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:47.353060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7f9d4-91ba-487b-aa98-f6751019db13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:47.353940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7f9d4-91ba-487b-aa98-f6751019db13 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:47.354228 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7f9d4-91ba-487b-aa98-f6751019db13 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:47.354509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7f9d4-91ba-487b-aa98-f6751019db13 None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:47.356214 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7f9d4-91ba-487b-aa98-f6751019db13 None admin] RBAC: Authorizing `identity:get_project(project_id=6525e5222e3342a3a26a6369518c7577)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:47.357949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7f9d4-91ba-487b-aa98-f6751019db13 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:47.359929 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 122/229] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:47 2022] GET /identity/v3/projects/6525e5222e3342a3a26a6369518c7577 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:47.368884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7f9d4-91ba-487b-aa98-f6751019db13 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:47.478557 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e212d-1aae-4b90-aeed-9c26f87889f3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:47.538884 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b7396-2ef1-47ae-ba1c-8bd099e91601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:47.539932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b7396-2ef1-47ae-ba1c-8bd099e91601 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:47.540444 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b7396-2ef1-47ae-ba1c-8bd099e91601 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:47.540931 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b7396-2ef1-47ae-ba1c-8bd099e91601 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:47.541880 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b7396-2ef1-47ae-ba1c-8bd099e91601 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:47.542389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b7396-2ef1-47ae-ba1c-8bd099e91601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:47.544091 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b7396-2ef1-47ae-ba1c-8bd099e91601 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:47.555558 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 108/230] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:22:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:47.559482 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b7396-2ef1-47ae-ba1c-8bd099e91601 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:47.572418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7f9d4-91ba-487b-aa98-f6751019db13 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:47.633130 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44009138-0177-49ea-9b8f-3946acd1d482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:47.634393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44009138-0177-49ea-9b8f-3946acd1d482 None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:47.634888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44009138-0177-49ea-9b8f-3946acd1d482 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:47.635396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44009138-0177-49ea-9b8f-3946acd1d482 None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577/users/ad2e3cebb6354f6aa4d0c19f24807e4e/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:47.639090 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44009138-0177-49ea-9b8f-3946acd1d482 None admin] RBAC: Authorizing `identity:create_grant(project_id=6525e5222e3342a3a26a6369518c7577, user_id=ad2e3cebb6354f6aa4d0c19f24807e4e, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:47.641882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44009138-0177-49ea-9b8f-3946acd1d482 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:47.664399 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 123/231] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:22:47 2022] PUT /identity/v3/projects/6525e5222e3342a3a26a6369518c7577/users/ad2e3cebb6354f6aa4d0c19f24807e4e/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:22:47.705962 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:47.715715 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b7396-2ef1-47ae-ba1c-8bd099e91601 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:47.754900 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c4004-beb6-498a-9d1b-af58efafd6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:47.756193 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c4004-beb6-498a-9d1b-af58efafd6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:47.756703 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c4004-beb6-498a-9d1b-af58efafd6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:47.757912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c4004-beb6-498a-9d1b-af58efafd6c4 None admin] PATH_INFO: `/v3/users/ad2e3cebb6354f6aa4d0c19f24807e4e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:47.759547 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c4004-beb6-498a-9d1b-af58efafd6c4 None admin] RBAC: Authorizing `identity:get_user(user_id=ad2e3cebb6354f6aa4d0c19f24807e4e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:47.761246 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c4004-beb6-498a-9d1b-af58efafd6c4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:47.763236 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 109/232] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:47 2022] GET /identity/v3/users/ad2e3cebb6354f6aa4d0c19f24807e4e => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:47.772344 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44009138-0177-49ea-9b8f-3946acd1d482 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:47.815977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:47.906689 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44009138-0177-49ea-9b8f-3946acd1d482 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:47.953449 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772b758-bda4-4a99-8012-75481943986f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:47.954338 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772b758-bda4-4a99-8012-75481943986f None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:47.954628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772b758-bda4-4a99-8012-75481943986f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:47.954913 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772b758-bda4-4a99-8012-75481943986f None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:47.956270 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b772b758-bda4-4a99-8012-75481943986f None admin] RBAC: Authorizing `identity:get_project(project_id=6525e5222e3342a3a26a6369518c7577)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:47.957757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b772b758-bda4-4a99-8012-75481943986f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:47.959514 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 124/233] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:47 2022] GET /identity/v3/projects/6525e5222e3342a3a26a6369518c7577 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:47.965269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c4004-beb6-498a-9d1b-af58efafd6c4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:48.114315 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c4004-beb6-498a-9d1b-af58efafd6c4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:48.165339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b0208-51fd-4eb3-a568-4dac7b33ca89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:48.166311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b0208-51fd-4eb3-a568-4dac7b33ca89 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:48.166644 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b0208-51fd-4eb3-a568-4dac7b33ca89 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:48.166951 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b0208-51fd-4eb3-a568-4dac7b33ca89 None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577/users/ad2e3cebb6354f6aa4d0c19f24807e4e/roles/61397d4a1ccd4a8ca479f98186ed489b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:48.169188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b0208-51fd-4eb3-a568-4dac7b33ca89 None admin] RBAC: Authorizing `identity:create_grant(project_id=6525e5222e3342a3a26a6369518c7577, user_id=ad2e3cebb6354f6aa4d0c19f24807e4e, role_id=61397d4a1ccd4a8ca479f98186ed489b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:48.170636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b0208-51fd-4eb3-a568-4dac7b33ca89 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:48.203380 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 110/234] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:22:47 2022] PUT /identity/v3/projects/6525e5222e3342a3a26a6369518c7577/users/ad2e3cebb6354f6aa4d0c19f24807e4e/roles/61397d4a1ccd4a8ca479f98186ed489b => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:22:48.604860 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fc3a9-8806-490f-b3f8-0a90779f882f None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:48.605902 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fc3a9-8806-490f-b3f8-0a90779f882f None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:48.606400 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794fc3a9-8806-490f-b3f8-0a90779f882f None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:48.608318 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 125/235] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:48.613950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6c19a-2326-4baf-ae1c-904b22edca9d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:48.614410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6c19a-2326-4baf-ae1c-904b22edca9d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:48.614590 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6c19a-2326-4baf-ae1c-904b22edca9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:48.636806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-45b6c19a-2326-4baf-ae1c-904b22edca9d None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:48.638009 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-45b6c19a-2326-4baf-ae1c-904b22edca9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:48.660004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:48.736559 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 111/236] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:22:48.773377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794fc3a9-8806-490f-b3f8-0a90779f882f None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:48.880054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96def6-c371-4673-8f62-ace92d16b622 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:48.880218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96def6-c371-4673-8f62-ace92d16b622 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:48.880513 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e96def6-c371-4673-8f62-ace92d16b622 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:48.882012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 112/237] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:22:48.890603 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc65d3-d25b-4f50-8546-35f155d6e203 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:48.890904 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc65d3-d25b-4f50-8546-35f155d6e203 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:48.891249 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc65d3-d25b-4f50-8546-35f155d6e203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:48.897103 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:48.904899 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794fc3a9-8806-490f-b3f8-0a90779f882f None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:48.926209 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-57bc65d3-d25b-4f50-8546-35f155d6e203 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:48.927435 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-57bc65d3-d25b-4f50-8546-35f155d6e203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:48.963400 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27222eb3-a0ca-4457-8419-e866bc5f88d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:48.964294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27222eb3-a0ca-4457-8419-e866bc5f88d7 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:48.964621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27222eb3-a0ca-4457-8419-e866bc5f88d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:48.964907 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27222eb3-a0ca-4457-8419-e866bc5f88d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:48.985348 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27222eb3-a0ca-4457-8419-e866bc5f88d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:48.999244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-27222eb3-a0ca-4457-8419-e866bc5f88d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:22:48.999244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:48.999244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:48.999244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:48.999244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:48.999244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:48.999244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:48.999244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:48.999244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:48.999244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:48.999244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:48.999244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:48.999244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:48.999244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:48.999244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:48.999244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:49.000204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:49.000204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:49.000204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:49.000204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:49.000204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:49.000204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:49.000204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:49.000204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:49.000204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:49.000204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:49.000204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:49.000204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:49.000204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:49.000204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:49.000204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:49.000204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:49.001213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:49.001213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:49.001213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:49.001213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:49.001213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:49.001213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:49.001213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:49.001213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:49.001213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:49.001213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:49.001213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:22:49.001213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:49.001213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 126/238] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:49.008051 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27222eb3-a0ca-4457-8419-e866bc5f88d7 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:49.045461 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 113/239] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:22:49.080580 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc65d3-d25b-4f50-8546-35f155d6e203 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:49.199481 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27222eb3-a0ca-4457-8419-e866bc5f88d7 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:49.231337 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:49.240090 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89481824-579d-4e31-afdf-0b4baa88dfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:49.240955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89481824-579d-4e31-afdf-0b4baa88dfb5 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:49.241630 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89481824-579d-4e31-afdf-0b4baa88dfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:49.241740 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc65d3-d25b-4f50-8546-35f155d6e203 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:49.242440 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89481824-579d-4e31-afdf-0b4baa88dfb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:49.243716 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89481824-579d-4e31-afdf-0b4baa88dfb5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:49.244235 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89481824-579d-4e31-afdf-0b4baa88dfb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:49.245927 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89481824-579d-4e31-afdf-0b4baa88dfb5 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:49.254954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 127/240] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:49.256465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fde05e-b6b6-4e6b-a95c-5750a61d5b2a None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:49.257743 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fde05e-b6b6-4e6b-a95c-5750a61d5b2a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:49.258286 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fde05e-b6b6-4e6b-a95c-5750a61d5b2a None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:49.259784 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 128/241] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:49.261201 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fde05e-b6b6-4e6b-a95c-5750a61d5b2a None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:49.281696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:49.291950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627b0e6-7205-41c9-9e97-7171cc972109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:49.292871 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627b0e6-7205-41c9-9e97-7171cc972109 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:49.293175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627b0e6-7205-41c9-9e97-7171cc972109 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:49.293465 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627b0e6-7205-41c9-9e97-7171cc972109 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:49.294665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627b0e6-7205-41c9-9e97-7171cc972109 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:49.296128 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627b0e6-7205-41c9-9e97-7171cc972109 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:49.297761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 114/242] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:22:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:49.298898 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75fb65-81f0-4fdd-abf9-8c5aa436267a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:49.303334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75fb65-81f0-4fdd-abf9-8c5aa436267a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:49.304598 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75fb65-81f0-4fdd-abf9-8c5aa436267a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:49.325402 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bd75fb65-81f0-4fdd-abf9-8c5aa436267a None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:49.327154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bd75fb65-81f0-4fdd-abf9-8c5aa436267a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:49.392984 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fde05e-b6b6-4e6b-a95c-5750a61d5b2a None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:49.418944 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 115/243] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:49.418944 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75fb65-81f0-4fdd-abf9-8c5aa436267a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:49.450181 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b34e8-8af4-40b0-be22-d0d222f6c0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:49.451018 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b34e8-8af4-40b0-be22-d0d222f6c0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:49.451313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b34e8-8af4-40b0-be22-d0d222f6c0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:49.451604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b34e8-8af4-40b0-be22-d0d222f6c0d9 None admin] PATH_INFO: `/v3/users/cd8f2d3883334117899f753bede9c227` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:49.466458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b34e8-8af4-40b0-be22-d0d222f6c0d9 None admin] RBAC: Authorizing `identity:get_user(user_id=cd8f2d3883334117899f753bede9c227)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:49.467612 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b34e8-8af4-40b0-be22-d0d222f6c0d9 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:49.473904 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 129/244] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:49 2022] GET /identity/v3/users/cd8f2d3883334117899f753bede9c227 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:49.473904 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b34e8-8af4-40b0-be22-d0d222f6c0d9 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:49.524328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:49.549306 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75fb65-81f0-4fdd-abf9-8c5aa436267a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:49.558222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:49.584078 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b34e8-8af4-40b0-be22-d0d222f6c0d9 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:49.592249 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b1950-7c70-467b-9750-6680ad644644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:49.593450 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b1950-7c70-467b-9750-6680ad644644 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:49.593943 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b1950-7c70-467b-9750-6680ad644644 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:49.594426 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b1950-7c70-467b-9750-6680ad644644 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:49.595562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b1950-7c70-467b-9750-6680ad644644 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:49.597414 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b1950-7c70-467b-9750-6680ad644644 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:49.626952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 116/245] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 22:22:49 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:22:49.626952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b1950-7c70-467b-9750-6680ad644644 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:49.633333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f51124-65e8-4df4-ad09-9e8354111541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:49.634232 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f51124-65e8-4df4-ad09-9e8354111541 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:49.634529 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f51124-65e8-4df4-ad09-9e8354111541 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:49.634862 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f51124-65e8-4df4-ad09-9e8354111541 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:49.660189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f51124-65e8-4df4-ad09-9e8354111541 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:49.676223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-40f51124-65e8-4df4-ad09-9e8354111541 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:49.676223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:49.676223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:49.676223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:49.676223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:49.676223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:49.676223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:49.676223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:49.676223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:49.676223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:49.676223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:49.676223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:49.676223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:49.676223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:49.676223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:49.676223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:49.677215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:49.677215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:49.677215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:49.677215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:49.677215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:49.677215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:49.677215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:49.677215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:49.677215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:49.677215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:49.677215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:49.677215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:49.677215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:49.677215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:49.677215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:49.677215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:49.678043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:49.678043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:49.678043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:49.678043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:49.678043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:49.678043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:49.678043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:49.678043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:49.678043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:49.678043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:49.678043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:49.678043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:49.682829 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 130/246] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:49.688986 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f51124-65e8-4df4-ad09-9e8354111541 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:49.855743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b1950-7c70-467b-9750-6680ad644644 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:49.861886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f51124-65e8-4df4-ad09-9e8354111541 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:49.907199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf21b2-3554-43fe-95d9-2bafb28ef348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:49.908253 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf21b2-3554-43fe-95d9-2bafb28ef348 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:49.908906 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf21b2-3554-43fe-95d9-2bafb28ef348 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:49.909196 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf21b2-3554-43fe-95d9-2bafb28ef348 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:49.913742 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:49.917028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80221d80-9eda-4bee-9e59-d9fa4f5e58ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:49.918821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80221d80-9eda-4bee-9e59-d9fa4f5e58ef None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:49.919143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80221d80-9eda-4bee-9e59-d9fa4f5e58ef None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:49.919434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80221d80-9eda-4bee-9e59-d9fa4f5e58ef None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:49.920346 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80221d80-9eda-4bee-9e59-d9fa4f5e58ef None admin] RBAC: Adding query filter params (name=member) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:49.920723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80221d80-9eda-4bee-9e59-d9fa4f5e58ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:49.922307 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80221d80-9eda-4bee-9e59-d9fa4f5e58ef None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:49.925226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:49.938764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 131/247] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:49.944150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80221d80-9eda-4bee-9e59-d9fa4f5e58ef None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:50.016686 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf21b2-3554-43fe-95d9-2bafb28ef348 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:50.020301 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf21b2-3554-43fe-95d9-2bafb28ef348 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:50.075408 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80221d80-9eda-4bee-9e59-d9fa4f5e58ef None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:50.117259 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ee7cf-cab7-440f-ac4d-e92a9940fd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:50.118133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ee7cf-cab7-440f-ac4d-e92a9940fd7f None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:50.118435 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ee7cf-cab7-440f-ac4d-e92a9940fd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:50.118721 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ee7cf-cab7-440f-ac4d-e92a9940fd7f None admin] PATH_INFO: `/v3/users/7faa873fa2c34876a03eed73038a6dd4` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:50.121068 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ee7cf-cab7-440f-ac4d-e92a9940fd7f None admin] RBAC: Authorizing `identity:get_user(user_id=7faa873fa2c34876a03eed73038a6dd4)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:50.122281 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ee7cf-cab7-440f-ac4d-e92a9940fd7f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:50.124563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 132/248] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:49 2022] GET /identity/v3/users/7faa873fa2c34876a03eed73038a6dd4 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:50.125842 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ae385-0665-4c6c-8b7f-e7eea757cc29 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:50.126136 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ae385-0665-4c6c-8b7f-e7eea757cc29 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:50.126470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ae385-0665-4c6c-8b7f-e7eea757cc29 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:50.136107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 133/249] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:50.136107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ae385-0665-4c6c-8b7f-e7eea757cc29 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:50.165695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 117/250] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:22:50.167802 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331cf55-154d-4a2c-86a0-79c487ce4238 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:50.168306 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331cf55-154d-4a2c-86a0-79c487ce4238 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:50.168846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331cf55-154d-4a2c-86a0-79c487ce4238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:50.192311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d331cf55-154d-4a2c-86a0-79c487ce4238 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:50.193850 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d331cf55-154d-4a2c-86a0-79c487ce4238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:50.227733 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:50.239888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:50.287232 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ae385-0665-4c6c-8b7f-e7eea757cc29 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:50.290370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 118/251] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:50.294974 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331cf55-154d-4a2c-86a0-79c487ce4238 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:50.355931 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf94cb5-d0ef-4023-b5fd-afaa4db95d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:50.357093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf94cb5-d0ef-4023-b5fd-afaa4db95d91 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:50.357622 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf94cb5-d0ef-4023-b5fd-afaa4db95d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:50.358047 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf94cb5-d0ef-4023-b5fd-afaa4db95d91 None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:50.361557 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf94cb5-d0ef-4023-b5fd-afaa4db95d91 None admin] RBAC: Authorizing `identity:get_project(project_id=6525e5222e3342a3a26a6369518c7577)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:50.363246 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf94cb5-d0ef-4023-b5fd-afaa4db95d91 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:50.365185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 134/252] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:50 2022] GET /identity/v3/projects/6525e5222e3342a3a26a6369518c7577 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:50.368783 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf94cb5-d0ef-4023-b5fd-afaa4db95d91 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:50.477771 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331cf55-154d-4a2c-86a0-79c487ce4238 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:50.545638 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf94cb5-d0ef-4023-b5fd-afaa4db95d91 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:50.548316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665f15c-34c7-41f0-bf3c-941ffa545b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:50.549576 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665f15c-34c7-41f0-bf3c-941ffa545b85 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:50.550083 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665f15c-34c7-41f0-bf3c-941ffa545b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:50.550586 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665f15c-34c7-41f0-bf3c-941ffa545b85 None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:50.552063 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665f15c-34c7-41f0-bf3c-941ffa545b85 None admin] RBAC: Authorizing `identity:get_project(project_id=6525e5222e3342a3a26a6369518c7577)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:50.553796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665f15c-34c7-41f0-bf3c-941ffa545b85 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:50.555917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 119/253] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:50 2022] GET /identity/v3/projects/6525e5222e3342a3a26a6369518c7577 => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:50.556884 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665f15c-34c7-41f0-bf3c-941ffa545b85 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:50.596024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:50.644485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53685944-9c6e-4613-83d9-96884e3f5f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:50.645945 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53685944-9c6e-4613-83d9-96884e3f5f0b None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:50.646456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53685944-9c6e-4613-83d9-96884e3f5f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:50.647248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53685944-9c6e-4613-83d9-96884e3f5f0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:50.649956 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:50.665752 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53685944-9c6e-4613-83d9-96884e3f5f0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:50.675194 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665f15c-34c7-41f0-bf3c-941ffa545b85 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:50.677145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-53685944-9c6e-4613-83d9-96884e3f5f0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:22:50.677145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:50.677145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:50.677145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:50.677145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:50.677145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:50.677145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:50.677145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:50.677145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:50.677145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:50.677145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:50.677145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:50.677145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:50.677145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:50.677145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:50.677145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:50.678007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:50.678007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:50.678007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:50.678007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:50.678007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:50.678007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:50.678007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:50.678007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:50.678007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:50.678007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:50.678007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:50.678007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:50.678007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:50.678007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:50.678007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:50.678007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:50.679563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:50.679563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:50.679563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:50.679563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:50.679563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:50.679563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:50.679563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:50.679563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:50.679563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:50.679563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:50.679563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:22:50.679563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:50.681258 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 135/254] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:22:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:50.683793 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53685944-9c6e-4613-83d9-96884e3f5f0b None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:50.713243 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d8447-fd27-4ad6-9e2d-bcc3ff6d778c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:50.714139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d8447-fd27-4ad6-9e2d-bcc3ff6d778c None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:50.714515 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d8447-fd27-4ad6-9e2d-bcc3ff6d778c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:50.714704 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d8447-fd27-4ad6-9e2d-bcc3ff6d778c None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577/users/7faa873fa2c34876a03eed73038a6dd4/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:50.716844 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d8447-fd27-4ad6-9e2d-bcc3ff6d778c None admin] RBAC: Authorizing `identity:create_grant(project_id=6525e5222e3342a3a26a6369518c7577, user_id=7faa873fa2c34876a03eed73038a6dd4, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:50.718291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d8447-fd27-4ad6-9e2d-bcc3ff6d778c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:50.732894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 120/255] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:22:50 2022] PUT /identity/v3/projects/6525e5222e3342a3a26a6369518c7577/users/7faa873fa2c34876a03eed73038a6dd4/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:22:50.733735 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a361c5-31f6-4cdb-921b-1479b310b14a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:50.734036 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a361c5-31f6-4cdb-921b-1479b310b14a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:50.734320 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a361c5-31f6-4cdb-921b-1479b310b14a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:50.735754 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 121/256] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:50.740670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a361c5-31f6-4cdb-921b-1479b310b14a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:50.827274 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53685944-9c6e-4613-83d9-96884e3f5f0b None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:50.870637 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710faf9-afe5-45cc-bfd2-6b7a741fba98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:50.871751 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710faf9-afe5-45cc-bfd2-6b7a741fba98 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:50.872250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710faf9-afe5-45cc-bfd2-6b7a741fba98 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:50.872999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710faf9-afe5-45cc-bfd2-6b7a741fba98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:50.874637 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710faf9-afe5-45cc-bfd2-6b7a741fba98 None admin] RBAC: Adding query filter params (role.id=ff3357541f744b6094e8e5fb5a79aa32, scope.project.id=6525e5222e3342a3a26a6369518c7577, user.id=cd8f2d3883334117899f753bede9c227) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:50.875356 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710faf9-afe5-45cc-bfd2-6b7a741fba98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:50.877148 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710faf9-afe5-45cc-bfd2-6b7a741fba98 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:50.884975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 136/257] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:22:50 2022] GET /identity/v3/role_assignments?user.id=cd8f2d3883334117899f753bede9c227&scope.project.id=6525e5222e3342a3a26a6369518c7577&role.id=ff3357541f744b6094e8e5fb5a79aa32 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:50.886619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e76b04-8c57-4981-b416-36c308ba013e None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:50.891797 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e76b04-8c57-4981-b416-36c308ba013e None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:50.892423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e76b04-8c57-4981-b416-36c308ba013e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:50.918077 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-f9e76b04-8c57-4981-b416-36c308ba013e None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:50.921052 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-f9e76b04-8c57-4981-b416-36c308ba013e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:51.009434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:51.013102 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:51.030499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a361c5-31f6-4cdb-921b-1479b310b14a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:51.045731 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 137/258] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:51.074741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945f523-f955-4e3e-9ceb-b28e151ef8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:51.075457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945f523-f955-4e3e-9ceb-b28e151ef8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:51.075749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945f523-f955-4e3e-9ceb-b28e151ef8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:51.076091 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945f523-f955-4e3e-9ceb-b28e151ef8f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:51.076912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945f523-f955-4e3e-9ceb-b28e151ef8f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:51.077230 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945f523-f955-4e3e-9ceb-b28e151ef8f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:51.078666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945f523-f955-4e3e-9ceb-b28e151ef8f7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:51.088873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 122/259] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:51.096306 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e76b04-8c57-4981-b416-36c308ba013e None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:51.100326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945f523-f955-4e3e-9ceb-b28e151ef8f7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:51.242823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e76b04-8c57-4981-b416-36c308ba013e None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:51.291127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945f523-f955-4e3e-9ceb-b28e151ef8f7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:51.294888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93695422-b50a-4768-bbb2-7d811f3d645b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:51.295983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93695422-b50a-4768-bbb2-7d811f3d645b None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:51.296464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93695422-b50a-4768-bbb2-7d811f3d645b None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:51.296932 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93695422-b50a-4768-bbb2-7d811f3d645b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:51.331052 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93695422-b50a-4768-bbb2-7d811f3d645b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:51.342703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-93695422-b50a-4768-bbb2-7d811f3d645b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:22:51.342703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:51.342703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:51.342703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:51.342703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:51.342703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:51.342703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:51.342703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:51.342703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:51.342703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:51.342703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:51.342703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:51.342703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:51.342703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:51.342703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:51.342703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:51.343605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:51.343605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:51.343605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:51.343605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:51.343605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:51.343605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:51.343605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:51.343605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:51.343605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:51.343605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:51.343605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:51.343605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:51.343605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:51.343605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:51.343605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:51.343605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:51.344394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:51.344394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:51.344394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:51.344394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:51.344394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:51.344394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:51.344394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:51.344394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:51.344394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:51.344394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:51.344394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:22:51.344394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:51.352460 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 138/260] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:22:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:51.361201 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93695422-b50a-4768-bbb2-7d811f3d645b None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:51.365151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1b47f-701a-46c4-9a69-1354fae3ca60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:51.367297 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1b47f-701a-46c4-9a69-1354fae3ca60 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:51.367588 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1b47f-701a-46c4-9a69-1354fae3ca60 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:51.367872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1b47f-701a-46c4-9a69-1354fae3ca60 None admin] PATH_INFO: `/v3/users/ad2e3cebb6354f6aa4d0c19f24807e4e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:51.370384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1b47f-701a-46c4-9a69-1354fae3ca60 None admin] RBAC: Authorizing `identity:get_user(user_id=ad2e3cebb6354f6aa4d0c19f24807e4e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:51.380496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1b47f-701a-46c4-9a69-1354fae3ca60 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:51.383361 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 123/261] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:51 2022] GET /identity/v3/users/ad2e3cebb6354f6aa4d0c19f24807e4e => generated 283 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:51.389231 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1b47f-701a-46c4-9a69-1354fae3ca60 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:51.448516 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:51.496498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:51.605185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93695422-b50a-4768-bbb2-7d811f3d645b None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:51.667816 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1b47f-701a-46c4-9a69-1354fae3ca60 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:51.689002 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32903b96-09aa-4aa0-9867-60ec402a3833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:51.690246 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32903b96-09aa-4aa0-9867-60ec402a3833 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:51.690683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32903b96-09aa-4aa0-9867-60ec402a3833 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:51.691130 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32903b96-09aa-4aa0-9867-60ec402a3833 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:51.692064 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32903b96-09aa-4aa0-9867-60ec402a3833 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:51.692529 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32903b96-09aa-4aa0-9867-60ec402a3833 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:51.698943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32903b96-09aa-4aa0-9867-60ec402a3833 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:51.713765 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06db64-44aa-41c3-b8fb-27d891134001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:51.714422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06db64-44aa-41c3-b8fb-27d891134001 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:51.714767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06db64-44aa-41c3-b8fb-27d891134001 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:51.715123 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06db64-44aa-41c3-b8fb-27d891134001 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:51.715274 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 139/262] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:22:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:51.720318 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32903b96-09aa-4aa0-9867-60ec402a3833 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:51.835412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06db64-44aa-41c3-b8fb-27d891134001 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:51.836854 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06db64-44aa-41c3-b8fb-27d891134001 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:51.886945 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:51.975043 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 124/263] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:22:51.983281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06db64-44aa-41c3-b8fb-27d891134001 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:51.993839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32903b96-09aa-4aa0-9867-60ec402a3833 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:52.025091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:52.103739 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718f2c6-d136-48a1-84b5-a0db57a7c58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:52.105004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718f2c6-d136-48a1-84b5-a0db57a7c58e None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:52.105491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718f2c6-d136-48a1-84b5-a0db57a7c58e None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:52.105968 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718f2c6-d136-48a1-84b5-a0db57a7c58e None admin] PATH_INFO: `/v3/users/ad2e3cebb6354f6aa4d0c19f24807e4e` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:52.107468 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718f2c6-d136-48a1-84b5-a0db57a7c58e None admin] RBAC: Authorizing `identity:get_user(user_id=ad2e3cebb6354f6aa4d0c19f24807e4e)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:52.109127 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718f2c6-d136-48a1-84b5-a0db57a7c58e None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:52.111306 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 140/264] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:51 2022] GET /identity/v3/users/ad2e3cebb6354f6aa4d0c19f24807e4e => generated 283 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:52.132304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718f2c6-d136-48a1-84b5-a0db57a7c58e None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:52.195017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06db64-44aa-41c3-b8fb-27d891134001 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:52.252576 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:52.287633 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff5200-1c2c-4b18-ac02-d826dd374536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:52.289051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff5200-1c2c-4b18-ac02-d826dd374536 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:52.289338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff5200-1c2c-4b18-ac02-d826dd374536 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:52.289619 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff5200-1c2c-4b18-ac02-d826dd374536 None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:52.290853 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff5200-1c2c-4b18-ac02-d826dd374536 None admin] RBAC: Authorizing `identity:get_project(project_id=6525e5222e3342a3a26a6369518c7577)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:52.292326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff5200-1c2c-4b18-ac02-d826dd374536 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:52.294527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 125/265] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:51 2022] GET /identity/v3/projects/6525e5222e3342a3a26a6369518c7577 => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:52.305647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff5200-1c2c-4b18-ac02-d826dd374536 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:52.329391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718f2c6-d136-48a1-84b5-a0db57a7c58e None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:52.400999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed6e32-0af6-40da-b028-8fcd5fe122b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:52.401593 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed6e32-0af6-40da-b028-8fcd5fe122b5 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:52.402073 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed6e32-0af6-40da-b028-8fcd5fe122b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:52.402363 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed6e32-0af6-40da-b028-8fcd5fe122b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:52.491371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:52.531293 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed6e32-0af6-40da-b028-8fcd5fe122b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:52.532768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed6e32-0af6-40da-b028-8fcd5fe122b5 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:52.538591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff5200-1c2c-4b18-ac02-d826dd374536 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:52.606085 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4f022-1241-4dfd-b13b-fec4b3d5cb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:52.606747 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4f022-1241-4dfd-b13b-fec4b3d5cb5f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:52.607164 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4f022-1241-4dfd-b13b-fec4b3d5cb5f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:52.607446 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4f022-1241-4dfd-b13b-fec4b3d5cb5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:52.608456 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4f022-1241-4dfd-b13b-fec4b3d5cb5f None admin] RBAC: Adding query filter params (role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.project.id=6525e5222e3342a3a26a6369518c7577, user.id=ad2e3cebb6354f6aa4d0c19f24807e4e) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:52.608773 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4f022-1241-4dfd-b13b-fec4b3d5cb5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:52.610413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4f022-1241-4dfd-b13b-fec4b3d5cb5f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:52.627301 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 126/266] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:22:52 2022] GET /identity/v3/role_assignments?user.id=ad2e3cebb6354f6aa4d0c19f24807e4e&scope.project.id=6525e5222e3342a3a26a6369518c7577&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 608 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:52.628545 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7cc32-c06e-413f-92c2-f769f8783533 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:52.628843 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7cc32-c06e-413f-92c2-f769f8783533 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:52.629226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7cc32-c06e-413f-92c2-f769f8783533 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:52.630645 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 127/267] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:52.636347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf923786-a9ff-4994-98b4-8ea64ca097a0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:52.636636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf923786-a9ff-4994-98b4-8ea64ca097a0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:52.636923 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf923786-a9ff-4994-98b4-8ea64ca097a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:52.670436 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 141/268] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:22:52.675386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bf923786-a9ff-4994-98b4-8ea64ca097a0 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:52.675768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed6e32-0af6-40da-b028-8fcd5fe122b5 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:52.676593 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bf923786-a9ff-4994-98b4-8ea64ca097a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:52.771677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:52.785578 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:52.799426 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed6e32-0af6-40da-b028-8fcd5fe122b5 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:52.815326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 128/269] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 23 22:22:52.817028 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa1f3e-12f1-49f0-94e6-835d54f5abee None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:52.817521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa1f3e-12f1-49f0-94e6-835d54f5abee None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:52.818001 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa1f3e-12f1-49f0-94e6-835d54f5abee None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:52.820149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 129/270] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:52.825661 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf712a-f86b-4400-8611-50767f8ad476 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:52.826190 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf712a-f86b-4400-8611-50767f8ad476 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:52.826693 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf712a-f86b-4400-8611-50767f8ad476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:52.842902 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe9e9d-4027-4a14-b6fa-ff6fcc12d182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:52.843806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe9e9d-4027-4a14-b6fa-ff6fcc12d182 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:52.844097 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe9e9d-4027-4a14-b6fa-ff6fcc12d182 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:52.844390 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe9e9d-4027-4a14-b6fa-ff6fcc12d182 None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:52.845576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe9e9d-4027-4a14-b6fa-ff6fcc12d182 None admin] RBAC: Authorizing `identity:get_project(project_id=6525e5222e3342a3a26a6369518c7577)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:52.847177 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bcdf712a-f86b-4400-8611-50767f8ad476 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:52.847286 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe9e9d-4027-4a14-b6fa-ff6fcc12d182 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:52.848660 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bcdf712a-f86b-4400-8611-50767f8ad476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:52.848962 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 142/271] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:52 2022] GET /identity/v3/projects/6525e5222e3342a3a26a6369518c7577 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:52.856869 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe9e9d-4027-4a14-b6fa-ff6fcc12d182 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:52.939034 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 130/272] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:52.940364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf712a-f86b-4400-8611-50767f8ad476 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:53.016876 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe9e9d-4027-4a14-b6fa-ff6fcc12d182 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:53.067168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70eb017-76cd-44fd-a206-de6e3ebb3216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:53.068123 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eb017-76cd-44fd-a206-de6e3ebb3216 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:53.068838 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eb017-76cd-44fd-a206-de6e3ebb3216 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:53.069894 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eb017-76cd-44fd-a206-de6e3ebb3216 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:53.070942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70eb017-76cd-44fd-a206-de6e3ebb3216 None admin] RBAC: Adding query filter params (role.id=61397d4a1ccd4a8ca479f98186ed489b, scope.project.id=6525e5222e3342a3a26a6369518c7577, user.id=ad2e3cebb6354f6aa4d0c19f24807e4e) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:53.071680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70eb017-76cd-44fd-a206-de6e3ebb3216 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:53.074051 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70eb017-76cd-44fd-a206-de6e3ebb3216 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:53.087117 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 143/273] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:22:52 2022] GET /identity/v3/role_assignments?user.id=ad2e3cebb6354f6aa4d0c19f24807e4e&scope.project.id=6525e5222e3342a3a26a6369518c7577&role.id=61397d4a1ccd4a8ca479f98186ed489b => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:53.087117 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70eb017-76cd-44fd-a206-de6e3ebb3216 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:53.109179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf712a-f86b-4400-8611-50767f8ad476 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:53.118732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:53.143318 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:53.195675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1724e49-ea8f-4525-ae3e-a49f3dbd002b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:53.197312 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1724e49-ea8f-4525-ae3e-a49f3dbd002b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:53.197797 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1724e49-ea8f-4525-ae3e-a49f3dbd002b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:53.200305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1724e49-ea8f-4525-ae3e-a49f3dbd002b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:53.224005 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1724e49-ea8f-4525-ae3e-a49f3dbd002b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:53.238171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e1724e49-ea8f-4525-ae3e-a49f3dbd002b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:53.238171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:53.238171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:53.238171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:53.238171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:53.238171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:53.238171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:53.238171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:53.238171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:53.238171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:53.238171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:53.238171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:53.238171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:53.238171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:53.238171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:53.238171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:53.239352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:53.239352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:53.239352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:53.239352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:53.239352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:53.239352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:53.239352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:53.239352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:53.239352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:53.239352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:53.239352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:53.239352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:53.239352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:53.239352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:53.239352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:53.239352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:53.240321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:53.240321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:53.240321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:53.240321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:53.240321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:53.240321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:53.240321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:53.240321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:53.240321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:53.240321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:53.240321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:53.240321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:22:53.245318 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 131/274] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:53.251072 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1724e49-ea8f-4525-ae3e-a49f3dbd002b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:53.338477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70eb017-76cd-44fd-a206-de6e3ebb3216 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:53.391468 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17122ec-d313-4b50-8cc7-9f85ee0a005e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:53.393199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17122ec-d313-4b50-8cc7-9f85ee0a005e None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:53.393642 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17122ec-d313-4b50-8cc7-9f85ee0a005e None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:53.394105 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17122ec-d313-4b50-8cc7-9f85ee0a005e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:53.395784 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17122ec-d313-4b50-8cc7-9f85ee0a005e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:53.398943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17122ec-d313-4b50-8cc7-9f85ee0a005e None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:53.401432 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 144/275] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:22:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:53.407085 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17122ec-d313-4b50-8cc7-9f85ee0a005e None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:53.419567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1724e49-ea8f-4525-ae3e-a49f3dbd002b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:53.475323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de5e17-b9c3-4646-88f5-0d8f02458e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:53.476765 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de5e17-b9c3-4646-88f5-0d8f02458e62 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:53.477445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de5e17-b9c3-4646-88f5-0d8f02458e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:53.477803 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de5e17-b9c3-4646-88f5-0d8f02458e62 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:53.480597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de5e17-b9c3-4646-88f5-0d8f02458e62 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:53.481004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de5e17-b9c3-4646-88f5-0d8f02458e62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:53.483088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de5e17-b9c3-4646-88f5-0d8f02458e62 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:53.499842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 132/276] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:53.506221 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de5e17-b9c3-4646-88f5-0d8f02458e62 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:53.525413 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:53.550406 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17122ec-d313-4b50-8cc7-9f85ee0a005e None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:53.552946 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:53.588267 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f6e04-920a-41a5-ae3a-0d54934d474d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:53.589201 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f6e04-920a-41a5-ae3a-0d54934d474d None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:53.589634 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f6e04-920a-41a5-ae3a-0d54934d474d None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:53.590088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f6e04-920a-41a5-ae3a-0d54934d474d None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:53.590963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f6e04-920a-41a5-ae3a-0d54934d474d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:53.592553 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f6e04-920a-41a5-ae3a-0d54934d474d None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:53.613007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 145/277] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 22:22:53 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:22:53.614523 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c62f37-4e5b-4174-bab6-599c2419e406 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:53.615014 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c62f37-4e5b-4174-bab6-599c2419e406 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:53.615744 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c62f37-4e5b-4174-bab6-599c2419e406 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:53.617840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 146/278] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:53.626270 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92593dbb-08d0-4c57-b91d-fff0351d531b None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:53.626799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92593dbb-08d0-4c57-b91d-fff0351d531b None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:53.627338 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92593dbb-08d0-4c57-b91d-fff0351d531b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:53.639746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de5e17-b9c3-4646-88f5-0d8f02458e62 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:53.651632 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-92593dbb-08d0-4c57-b91d-fff0351d531b None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:53.653608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-92593dbb-08d0-4c57-b91d-fff0351d531b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:53.681871 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fda0a-b1ab-4256-bd4f-f847374c1bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:53.682996 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fda0a-b1ab-4256-bd4f-f847374c1bec None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:53.683389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fda0a-b1ab-4256-bd4f-f847374c1bec None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:53.683825 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fda0a-b1ab-4256-bd4f-f847374c1bec None admin] PATH_INFO: `/v3/users/dc76f6fb1e774047827d88badcebc53a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:53.700696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fda0a-b1ab-4256-bd4f-f847374c1bec None admin] RBAC: Authorizing `identity:get_user(user_id=dc76f6fb1e774047827d88badcebc53a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:53.702419 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fda0a-b1ab-4256-bd4f-f847374c1bec None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:53.704722 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 133/279] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:53 2022] GET /identity/v3/users/dc76f6fb1e774047827d88badcebc53a => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:53.710375 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fda0a-b1ab-4256-bd4f-f847374c1bec None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:53.751035 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 147/280] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:22:53.776211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92593dbb-08d0-4c57-b91d-fff0351d531b None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:53.835751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:53.843794 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fda0a-b1ab-4256-bd4f-f847374c1bec None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:53.849678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:53.887561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451cb9b-40a7-4390-8d72-09576f40b732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:53.888375 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451cb9b-40a7-4390-8d72-09576f40b732 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:53.888792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451cb9b-40a7-4390-8d72-09576f40b732 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:53.889083 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451cb9b-40a7-4390-8d72-09576f40b732 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:53.889918 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451cb9b-40a7-4390-8d72-09576f40b732 None admin] RBAC: Adding query filter params (role.id=1baba2239751440eb25af0d0fd2d072d, user.id=dc76f6fb1e774047827d88badcebc53a, scope.system=all) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:53.890395 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451cb9b-40a7-4390-8d72-09576f40b732 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:53.892325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451cb9b-40a7-4390-8d72-09576f40b732 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:53.900792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 134/281] 10.222.0.43 () {56 vars in 1308 bytes} [Thu Jun 23 22:22:53 2022] GET /identity/v3/role_assignments?user.id=dc76f6fb1e774047827d88badcebc53a&scope.system=all&role.id=1baba2239751440eb25af0d0fd2d072d => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:53.901520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92593dbb-08d0-4c57-b91d-fff0351d531b None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:53.902079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd5a1c-96fa-49f4-8875-a5d6fd05c40d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:53.902369 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd5a1c-96fa-49f4-8875-a5d6fd05c40d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:53.902651 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd5a1c-96fa-49f4-8875-a5d6fd05c40d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:53.906956 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 135/282] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:53.910583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7df54b-1d24-4080-91e8-9de75819d60d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:53.910871 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7df54b-1d24-4080-91e8-9de75819d60d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:53.911183 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7df54b-1d24-4080-91e8-9de75819d60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:53.937447 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fb7df54b-1d24-4080-91e8-9de75819d60d None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:53.938851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fb7df54b-1d24-4080-91e8-9de75819d60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:53.955537 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09834651-413d-4f75-ac77-0f42c878b468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:53.956370 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09834651-413d-4f75-ac77-0f42c878b468 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:53.956661 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09834651-413d-4f75-ac77-0f42c878b468 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:53.956946 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09834651-413d-4f75-ac77-0f42c878b468 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:53.971704 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09834651-413d-4f75-ac77-0f42c878b468 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:53.983558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-09834651-413d-4f75-ac77-0f42c878b468 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:53.983558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:53.983558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:53.983558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:53.983558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:53.983558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:53.983558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:53.983558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:53.983558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:53.983558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:53.983558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:53.983558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:53.983558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:53.983558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:53.983558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:53.983558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:53.984568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:53.984568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:53.984568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:53.984568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:53.984568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:53.984568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:53.984568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:53.984568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:53.984568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:53.984568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:53.984568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:53.984568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:53.984568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:53.984568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:53.984568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:53.984568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:53.985361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:53.985361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:53.985361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:53.985361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:53.985361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:53.985361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:53.985361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:53.985361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:53.985361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:53.985361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:53.985361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:53.985361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:53.987021 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 148/283] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:53.990964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09834651-413d-4f75-ac77-0f42c878b468 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:54.041328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 136/284] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:22:54.067853 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7df54b-1d24-4080-91e8-9de75819d60d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:54.119515 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09834651-413d-4f75-ac77-0f42c878b468 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:54.144301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:54.156854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7d37b-a35f-4898-b8a5-057a948279e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:54.157895 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7d37b-a35f-4898-b8a5-057a948279e1 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:54.158322 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7d37b-a35f-4898-b8a5-057a948279e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:54.158765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7d37b-a35f-4898-b8a5-057a948279e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:54.159689 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7d37b-a35f-4898-b8a5-057a948279e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:54.160138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7d37b-a35f-4898-b8a5-057a948279e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:54.161663 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7d37b-a35f-4898-b8a5-057a948279e1 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:54.172037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 149/285] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:54.178087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7d37b-a35f-4898-b8a5-057a948279e1 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:54.291295 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:54.314050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7df54b-1d24-4080-91e8-9de75819d60d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:54.326418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7d37b-a35f-4898-b8a5-057a948279e1 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:54.388374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ef71d-5931-474a-9450-c0a5ec994490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:54.389270 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ef71d-5931-474a-9450-c0a5ec994490 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:54.389561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ef71d-5931-474a-9450-c0a5ec994490 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:54.389847 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ef71d-5931-474a-9450-c0a5ec994490 None admin] PATH_INFO: `/v3/users/7faa873fa2c34876a03eed73038a6dd4` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:54.391546 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ef71d-5931-474a-9450-c0a5ec994490 None admin] RBAC: Authorizing `identity:get_user(user_id=7faa873fa2c34876a03eed73038a6dd4)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:54.393011 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ef71d-5931-474a-9450-c0a5ec994490 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:54.394651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 150/286] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:54 2022] GET /identity/v3/users/7faa873fa2c34876a03eed73038a6dd4 => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:54.395343 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53067ea8-93c4-4510-b642-2f75ff709611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:54.396150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53067ea8-93c4-4510-b642-2f75ff709611 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:54.396441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53067ea8-93c4-4510-b642-2f75ff709611 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:54.396725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53067ea8-93c4-4510-b642-2f75ff709611 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:54.403810 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ef71d-5931-474a-9450-c0a5ec994490 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:54.420758 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53067ea8-93c4-4510-b642-2f75ff709611 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:54.434189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-53067ea8-93c4-4510-b642-2f75ff709611 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:22:54.434189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:54.434189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:54.434189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:54.434189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:54.434189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:54.434189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:54.434189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:54.434189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:54.434189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:54.434189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:54.434189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:54.434189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:54.434189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:54.434189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:54.434189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:54.436613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:54.436613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:54.436613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:54.436613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:54.436613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:54.436613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:54.436613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:54.436613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:54.436613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:54.436613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:54.436613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:54.436613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:54.436613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:54.436613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:54.436613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:54.436613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:54.437787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:54.437787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:54.437787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:54.437787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:54.437787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:54.437787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:54.437787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:54.437787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:54.437787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:54.437787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:54.437787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:22:54.437787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:22:54.437787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 137/287] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:54.442455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53067ea8-93c4-4510-b642-2f75ff709611 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:54.538572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:54.595454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53067ea8-93c4-4510-b642-2f75ff709611 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:54.608061 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ef71d-5931-474a-9450-c0a5ec994490 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:54.631064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c48f24-c665-4298-b89f-0953667c32e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:54.632703 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c48f24-c665-4298-b89f-0953667c32e9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:54.633192 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c48f24-c665-4298-b89f-0953667c32e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:54.633667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c48f24-c665-4298-b89f-0953667c32e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:54.634615 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c48f24-c665-4298-b89f-0953667c32e9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:54.635148 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c48f24-c665-4298-b89f-0953667c32e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:54.636734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c48f24-c665-4298-b89f-0953667c32e9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:54.646064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 138/288] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:54.650258 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fefb464-d0da-4240-897f-d23f727cb493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:54.652014 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c48f24-c665-4298-b89f-0953667c32e9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:54.652754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefb464-d0da-4240-897f-d23f727cb493 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:54.653599 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefb464-d0da-4240-897f-d23f727cb493 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:54.662501 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fefb464-d0da-4240-897f-d23f727cb493 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:54.671260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:54.781662 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c48f24-c665-4298-b89f-0953667c32e9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:54.815405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ca826-fc10-46b9-b153-80bb817d3cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:54.816249 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ca826-fc10-46b9-b153-80bb817d3cbf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:54.816540 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ca826-fc10-46b9-b153-80bb817d3cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:54.816826 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ca826-fc10-46b9-b153-80bb817d3cbf None admin] PATH_INFO: `/v3/users/cd8f2d3883334117899f753bede9c227` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:54.818024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ca826-fc10-46b9-b153-80bb817d3cbf None admin] RBAC: Authorizing `identity:get_user(user_id=cd8f2d3883334117899f753bede9c227)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:54.819835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ca826-fc10-46b9-b153-80bb817d3cbf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:54.821872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 139/289] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:54 2022] GET /identity/v3/users/cd8f2d3883334117899f753bede9c227 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:22:54.829421 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ca826-fc10-46b9-b153-80bb817d3cbf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:54.849429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fefb464-d0da-4240-897f-d23f727cb493 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:54.851194 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fefb464-d0da-4240-897f-d23f727cb493 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:54.943264 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:54.979331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:54.987421 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ca826-fc10-46b9-b153-80bb817d3cbf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:55.048647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd76f4c-f40b-4fac-8b4d-868525531d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:55.049515 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd76f4c-f40b-4fac-8b4d-868525531d79 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:55.049800 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd76f4c-f40b-4fac-8b4d-868525531d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:55.050079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd76f4c-f40b-4fac-8b4d-868525531d79 None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:55.050633 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 151/290] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:22:55.051353 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd76f4c-f40b-4fac-8b4d-868525531d79 None admin] RBAC: Authorizing `identity:get_project(project_id=6525e5222e3342a3a26a6369518c7577)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:55.052823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd76f4c-f40b-4fac-8b4d-868525531d79 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:55.059266 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fefb464-d0da-4240-897f-d23f727cb493 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:55.062437 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 140/291] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:54 2022] GET /identity/v3/projects/6525e5222e3342a3a26a6369518c7577 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:55.067894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd76f4c-f40b-4fac-8b4d-868525531d79 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:55.226353 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd76f4c-f40b-4fac-8b4d-868525531d79 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:55.264143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fefb464-d0da-4240-897f-d23f727cb493 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:55.286061 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4a588-d4b3-46d6-a9da-03e05b82d611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:55.286969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4a588-d4b3-46d6-a9da-03e05b82d611 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:55.287263 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4a588-d4b3-46d6-a9da-03e05b82d611 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:55.287539 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4a588-d4b3-46d6-a9da-03e05b82d611 None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577/users/cd8f2d3883334117899f753bede9c227/roles/ff3357541f744b6094e8e5fb5a79aa32` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:55.291070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4a588-d4b3-46d6-a9da-03e05b82d611 None admin] RBAC: Authorizing `identity:create_grant(project_id=6525e5222e3342a3a26a6369518c7577, user_id=cd8f2d3883334117899f753bede9c227, role_id=ff3357541f744b6094e8e5fb5a79aa32)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:55.292504 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4a588-d4b3-46d6-a9da-03e05b82d611 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:55.313776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 141/292] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:22:55 2022] PUT /identity/v3/projects/6525e5222e3342a3a26a6369518c7577/users/cd8f2d3883334117899f753bede9c227/roles/ff3357541f744b6094e8e5fb5a79aa32 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:22:55.328432 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97130e23-6405-4d96-83e8-e05ed2c2da93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:55.329338 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97130e23-6405-4d96-83e8-e05ed2c2da93 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:55.329628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97130e23-6405-4d96-83e8-e05ed2c2da93 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:55.329909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97130e23-6405-4d96-83e8-e05ed2c2da93 None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:55.331304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97130e23-6405-4d96-83e8-e05ed2c2da93 None admin] RBAC: Authorizing `identity:get_project(project_id=6525e5222e3342a3a26a6369518c7577)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:55.332857 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97130e23-6405-4d96-83e8-e05ed2c2da93 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:55.335698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 152/293] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:55 2022] GET /identity/v3/projects/6525e5222e3342a3a26a6369518c7577 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:55.343613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4a588-d4b3-46d6-a9da-03e05b82d611 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:55.355309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:55.472897 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4a588-d4b3-46d6-a9da-03e05b82d611 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:55.508089 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a29fe0-8c18-4e22-93af-488145ac8208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:55.508763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a29fe0-8c18-4e22-93af-488145ac8208 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:55.509051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a29fe0-8c18-4e22-93af-488145ac8208 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:55.509339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a29fe0-8c18-4e22-93af-488145ac8208 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:55.510178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a29fe0-8c18-4e22-93af-488145ac8208 None admin] RBAC: Adding query filter params (role.id=1baba2239751440eb25af0d0fd2d072d, scope.project.id=6525e5222e3342a3a26a6369518c7577, user.id=7faa873fa2c34876a03eed73038a6dd4) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:55.510500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a29fe0-8c18-4e22-93af-488145ac8208 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:55.511983 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a29fe0-8c18-4e22-93af-488145ac8208 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:55.518604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 142/294] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:22:55 2022] GET /identity/v3/role_assignments?user.id=7faa873fa2c34876a03eed73038a6dd4&scope.project.id=6525e5222e3342a3a26a6369518c7577&role.id=1baba2239751440eb25af0d0fd2d072d => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:56.148095 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87f2cc-94c7-4f17-b91f-efe8fe7aa846 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:56.149057 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87f2cc-94c7-4f17-b91f-efe8fe7aa846 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:56.149539 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87f2cc-94c7-4f17-b91f-efe8fe7aa846 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:56.151143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 153/295] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:56.156618 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30302962-9c0e-4297-851d-b81469117046 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:56.157450 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30302962-9c0e-4297-851d-b81469117046 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:56.157954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30302962-9c0e-4297-851d-b81469117046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:56.176869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-30302962-9c0e-4297-851d-b81469117046 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:56.178378 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-30302962-9c0e-4297-851d-b81469117046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:56.250569 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:56.254063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56c375-b62b-4566-8dce-fc846e84ac5b None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:56.254622 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56c375-b62b-4566-8dce-fc846e84ac5b None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:56.255143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56c375-b62b-4566-8dce-fc846e84ac5b None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:56.257254 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 154/296] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:22:56.259730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 143/297] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:56.265257 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397b891-fcfd-4c5d-aea1-1aa422a55981 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:56.265779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397b891-fcfd-4c5d-aea1-1aa422a55981 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:56.266277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397b891-fcfd-4c5d-aea1-1aa422a55981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:56.269270 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3a8f7-b178-4f6a-abc6-5068678d831d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:56.269566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3a8f7-b178-4f6a-abc6-5068678d831d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:56.269851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3a8f7-b178-4f6a-abc6-5068678d831d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:56.271574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 144/298] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:22:56.278560 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964c0b1-3b3f-44ef-af25-1c92cd7f469e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:56.278854 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964c0b1-3b3f-44ef-af25-1c92cd7f469e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:56.279169 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964c0b1-3b3f-44ef-af25-1c92cd7f469e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:56.288376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-8397b891-fcfd-4c5d-aea1-1aa422a55981 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:56.289838 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8397b891-fcfd-4c5d-aea1-1aa422a55981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:56.296815 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0964c0b1-3b3f-44ef-af25-1c92cd7f469e None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:56.297959 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0964c0b1-3b3f-44ef-af25-1c92cd7f469e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:56.310301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:56.364979 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 155/299] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:56.371183 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 145/300] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:22:56.372271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397b891-fcfd-4c5d-aea1-1aa422a55981 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:56.388188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964c0b1-3b3f-44ef-af25-1c92cd7f469e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:56.478686 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397b891-fcfd-4c5d-aea1-1aa422a55981 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:56.489931 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964c0b1-3b3f-44ef-af25-1c92cd7f469e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:56.521141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338f941-6519-4855-9bf3-0a30998ec764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:56.522544 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338f941-6519-4855-9bf3-0a30998ec764 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:56.522824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338f941-6519-4855-9bf3-0a30998ec764 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:56.523152 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338f941-6519-4855-9bf3-0a30998ec764 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:56.524354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338f941-6519-4855-9bf3-0a30998ec764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:56.525895 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338f941-6519-4855-9bf3-0a30998ec764 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:56.527972 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 156/301] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:22:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:56.528555 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338f941-6519-4855-9bf3-0a30998ec764 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:56.531545 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:56.539269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce543d70-0c51-443f-82b9-fd789ec4d650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:56.540079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce543d70-0c51-443f-82b9-fd789ec4d650 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:56.540375 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce543d70-0c51-443f-82b9-fd789ec4d650 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:56.540664 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce543d70-0c51-443f-82b9-fd789ec4d650 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:56.551555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce543d70-0c51-443f-82b9-fd789ec4d650 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:56.552192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:56.560218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ce543d70-0c51-443f-82b9-fd789ec4d650 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:56.560218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:56.560218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:56.560218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:56.560218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:56.560218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:56.560218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:56.560218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:56.560218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:56.560218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:56.560218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:56.560218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:56.560218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:56.560218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:56.560218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:56.560218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:56.561095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:56.561095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:56.561095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:56.561095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:56.561095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:56.561095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:56.561095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:56.561095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:56.561095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:56.561095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:56.561095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:56.561095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:56.561095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:56.561095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:56.561095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:56.561095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:56.561872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:56.561872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:56.561872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:56.561872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:56.561872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:56.561872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:56.561872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:56.561872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:56.561872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:56.561872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:56.561872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:56.561872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:22:56.561872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 146/302] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:22:56.566960 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce543d70-0c51-443f-82b9-fd789ec4d650 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:56.639630 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338f941-6519-4855-9bf3-0a30998ec764 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:56.676305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a42781-41ae-4f27-b396-ef73975334df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:56.677111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a42781-41ae-4f27-b396-ef73975334df None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:56.677399 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a42781-41ae-4f27-b396-ef73975334df None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:56.677678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a42781-41ae-4f27-b396-ef73975334df None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:56.678099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce543d70-0c51-443f-82b9-fd789ec4d650 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:56.688780 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a42781-41ae-4f27-b396-ef73975334df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:56.696753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-75a42781-41ae-4f27-b396-ef73975334df None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:22:56.696753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:56.696753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:56.696753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:56.696753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:56.696753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:56.696753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:56.696753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:56.696753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:56.696753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:56.696753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:56.696753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:56.696753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:56.696753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:56.696753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:56.696753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:56.697820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:56.697820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:56.697820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:56.697820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:56.697820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:56.697820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:56.697820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:56.697820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:56.697820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:56.697820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:56.697820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:56.697820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:56.697820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:56.697820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:56.697820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:56.697820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:56.703511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:56.703511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:56.703511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:56.703511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:56.703511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:56.703511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:56.703511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:56.703511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:56.703511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:56.703511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:56.703511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:22:56.703511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:56.703511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 157/303] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:56.703511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a42781-41ae-4f27-b396-ef73975334df None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:56.719106 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f4b6e-98e7-4cc2-b44b-03eb5aaafa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:56.720035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f4b6e-98e7-4cc2-b44b-03eb5aaafa36 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:56.720321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f4b6e-98e7-4cc2-b44b-03eb5aaafa36 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:56.720602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f4b6e-98e7-4cc2-b44b-03eb5aaafa36 None admin] PATH_INFO: `/v3/groups` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:56.721287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f4b6e-98e7-4cc2-b44b-03eb5aaafa36 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:56.722715 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1f4b6e-98e7-4cc2-b44b-03eb5aaafa36 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:56.740347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 147/304] 10.222.0.43 () {60 vars in 1134 bytes} [Thu Jun 23 22:22:56 2022] POST /identity/v3/groups => generated 229 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:22:56.741069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f4b6e-98e7-4cc2-b44b-03eb5aaafa36 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:56.814278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:56.815914 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:56.845751 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a42781-41ae-4f27-b396-ef73975334df None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:56.879936 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f4b6e-98e7-4cc2-b44b-03eb5aaafa36 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:56.883798 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823f64a-4ae1-4442-82d0-88d5f74efb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:56.883798 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823f64a-4ae1-4442-82d0-88d5f74efb92 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:56.884065 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823f64a-4ae1-4442-82d0-88d5f74efb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:56.884278 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823f64a-4ae1-4442-82d0-88d5f74efb92 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:56.885105 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823f64a-4ae1-4442-82d0-88d5f74efb92 None admin] RBAC: Adding query filter params (name=member) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:56.885421 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823f64a-4ae1-4442-82d0-88d5f74efb92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:56.886883 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823f64a-4ae1-4442-82d0-88d5f74efb92 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:56.896354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 158/305] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:56.901991 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823f64a-4ae1-4442-82d0-88d5f74efb92 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:56.921628 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544ccba-e134-4dc5-8f1d-82069abd1f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:56.922668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544ccba-e134-4dc5-8f1d-82069abd1f39 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:56.923280 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544ccba-e134-4dc5-8f1d-82069abd1f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:56.923786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544ccba-e134-4dc5-8f1d-82069abd1f39 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:56.924862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544ccba-e134-4dc5-8f1d-82069abd1f39 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:56.925363 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544ccba-e134-4dc5-8f1d-82069abd1f39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:56.927041 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544ccba-e134-4dc5-8f1d-82069abd1f39 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:56.937024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 148/306] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:56.945652 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544ccba-e134-4dc5-8f1d-82069abd1f39 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:57.031176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823f64a-4ae1-4442-82d0-88d5f74efb92 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:57.067186 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544ccba-e134-4dc5-8f1d-82069abd1f39 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:57.074362 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af85bf6-206c-42ab-8276-46b9aca53b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:57.075279 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af85bf6-206c-42ab-8276-46b9aca53b4a None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:57.075987 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af85bf6-206c-42ab-8276-46b9aca53b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:57.076278 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af85bf6-206c-42ab-8276-46b9aca53b4a None admin] PATH_INFO: `/v3/users/dc76f6fb1e774047827d88badcebc53a` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:57.077865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af85bf6-206c-42ab-8276-46b9aca53b4a None admin] RBAC: Authorizing `identity:get_user(user_id=dc76f6fb1e774047827d88badcebc53a)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:57.079670 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af85bf6-206c-42ab-8276-46b9aca53b4a None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:57.081751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:57.081917 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 159/307] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:56 2022] GET /identity/v3/users/dc76f6fb1e774047827d88badcebc53a => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:57.088255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af85bf6-206c-42ab-8276-46b9aca53b4a None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:57.106154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:57.112095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318eee9-d7dc-47ad-ac12-fe354277e3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:57.113089 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318eee9-d7dc-47ad-ac12-fe354277e3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:57.113448 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318eee9-d7dc-47ad-ac12-fe354277e3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:57.113799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318eee9-d7dc-47ad-ac12-fe354277e3c7 None admin] PATH_INFO: `/v3/users/41aff0b978824cd389a2da81b4490e52` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:57.133798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318eee9-d7dc-47ad-ac12-fe354277e3c7 None admin] RBAC: Authorizing `identity:get_user(user_id=41aff0b978824cd389a2da81b4490e52)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:57.135793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318eee9-d7dc-47ad-ac12-fe354277e3c7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:57.138273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 149/308] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:56 2022] GET /identity/v3/users/41aff0b978824cd389a2da81b4490e52 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:57.146260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318eee9-d7dc-47ad-ac12-fe354277e3c7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:57.211449 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af85bf6-206c-42ab-8276-46b9aca53b4a None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:57.247783 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd9c6e-3008-47d2-8385-72e1363235cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:57.248682 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd9c6e-3008-47d2-8385-72e1363235cf None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:57.248970 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd9c6e-3008-47d2-8385-72e1363235cf None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:57.249253 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd9c6e-3008-47d2-8385-72e1363235cf None admin] PATH_INFO: `/v3/system/users/dc76f6fb1e774047827d88badcebc53a/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:57.250967 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd9c6e-3008-47d2-8385-72e1363235cf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=dc76f6fb1e774047827d88badcebc53a, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:57.252359 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd9c6e-3008-47d2-8385-72e1363235cf None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:57.262227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318eee9-d7dc-47ad-ac12-fe354277e3c7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:57.263566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 160/309] 10.222.0.43 () {58 vars in 1334 bytes} [Thu Jun 23 22:22:57 2022] PUT /identity/v3/system/users/dc76f6fb1e774047827d88badcebc53a/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:22:57.306051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da834266-50f9-4842-a703-2087c492da6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:57.306789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da834266-50f9-4842-a703-2087c492da6e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:57.307122 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da834266-50f9-4842-a703-2087c492da6e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:57.307405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da834266-50f9-4842-a703-2087c492da6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:57.308316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da834266-50f9-4842-a703-2087c492da6e None admin] RBAC: Adding query filter params (role.id=ff3357541f744b6094e8e5fb5a79aa32, user.id=41aff0b978824cd389a2da81b4490e52, scope.system=all) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:57.308702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da834266-50f9-4842-a703-2087c492da6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:57.310270 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da834266-50f9-4842-a703-2087c492da6e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:57.316769 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 150/310] 10.222.0.43 () {56 vars in 1308 bytes} [Thu Jun 23 22:22:57 2022] GET /identity/v3/role_assignments?user.id=41aff0b978824cd389a2da81b4490e52&scope.system=all&role.id=ff3357541f744b6094e8e5fb5a79aa32 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:57.588230 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345e912-6bef-4fa4-bb4c-e975d4a919ed None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:57.588630 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345e912-6bef-4fa4-bb4c-e975d4a919ed None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:57.588806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345e912-6bef-4fa4-bb4c-e975d4a919ed None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:57.590454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 161/311] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:57.595939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcb52a-9c25-4300-8ed0-df75a33abd82 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:57.596239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcb52a-9c25-4300-8ed0-df75a33abd82 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:57.596529 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcb52a-9c25-4300-8ed0-df75a33abd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:57.614330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b4fcb52a-9c25-4300-8ed0-df75a33abd82 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:57.615544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b4fcb52a-9c25-4300-8ed0-df75a33abd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:57.645221 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:57.691963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 151/312] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:22:57.716723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345e912-6bef-4fa4-bb4c-e975d4a919ed None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:57.800119 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:57.834153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345e912-6bef-4fa4-bb4c-e975d4a919ed None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:57.870516 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9323b2-be43-4418-a5cf-df869ce6e5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:57.871310 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9323b2-be43-4418-a5cf-df869ce6e5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:57.871595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9323b2-be43-4418-a5cf-df869ce6e5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:57.871879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9323b2-be43-4418-a5cf-df869ce6e5d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:57.882872 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9323b2-be43-4418-a5cf-df869ce6e5d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:57.890665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-de9323b2-be43-4418-a5cf-df869ce6e5d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:22:57.890665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:57.890665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:57.890665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:57.890665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:57.890665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:57.890665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:57.890665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:57.890665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:57.890665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:57.890665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:57.890665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:57.890665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:57.890665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:57.890665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:57.890665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:57.892723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:57.892723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:57.892723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:57.892723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:57.892723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:57.892723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:57.892723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:57.892723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:57.892723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:57.892723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:57.892723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:57.892723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:57.892723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:57.892723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:57.892723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:57.892723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:57.894604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:57.894604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:57.894604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:57.894604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:57.894604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:57.894604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:57.894604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:57.894604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:57.894604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:57.894604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:57.894604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:22:57.894604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:57.894604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 162/313] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:57.898953 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcb52a-9c25-4300-8ed0-df75a33abd82 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:58.008996 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcb52a-9c25-4300-8ed0-df75a33abd82 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:58.045712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fd40d-4705-49c5-82bd-daecafcc7a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:58.046385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fd40d-4705-49c5-82bd-daecafcc7a74 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:58.046674 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fd40d-4705-49c5-82bd-daecafcc7a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:58.046985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fd40d-4705-49c5-82bd-daecafcc7a74 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:58.047743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fd40d-4705-49c5-82bd-daecafcc7a74 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:58.048058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fd40d-4705-49c5-82bd-daecafcc7a74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:58.049444 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fd40d-4705-49c5-82bd-daecafcc7a74 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:58.059702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 152/314] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:58.066490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9323b2-be43-4418-a5cf-df869ce6e5d7 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:58.179855 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9323b2-be43-4418-a5cf-df869ce6e5d7 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:58.213299 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db7daf-ad7d-4e2e-9637-0822f0f75522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:58.214177 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db7daf-ad7d-4e2e-9637-0822f0f75522 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:58.214466 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db7daf-ad7d-4e2e-9637-0822f0f75522 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:58.214746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db7daf-ad7d-4e2e-9637-0822f0f75522 None admin] PATH_INFO: `/v3/users/cd8f2d3883334117899f753bede9c227` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:58.215972 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db7daf-ad7d-4e2e-9637-0822f0f75522 None admin] RBAC: Authorizing `identity:get_user(user_id=cd8f2d3883334117899f753bede9c227)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:58.217429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70db7daf-ad7d-4e2e-9637-0822f0f75522 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:58.219235 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 163/315] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:58 2022] GET /identity/v3/users/cd8f2d3883334117899f753bede9c227 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:58.224625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fd40d-4705-49c5-82bd-daecafcc7a74 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:58.254833 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:58.332171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fd40d-4705-49c5-82bd-daecafcc7a74 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:58.371524 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6886b13-a751-435d-988b-a518d93f8aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:58.372414 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6886b13-a751-435d-988b-a518d93f8aae None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:58.372870 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6886b13-a751-435d-988b-a518d93f8aae None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:58.373308 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6886b13-a751-435d-988b-a518d93f8aae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:58.449878 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6886b13-a751-435d-988b-a518d93f8aae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:58.451592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6886b13-a751-435d-988b-a518d93f8aae None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:58.481200 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:58.562897 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 153/316] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:22:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:22:58.568624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db7daf-ad7d-4e2e-9637-0822f0f75522 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:58.587288 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:58.684099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70db7daf-ad7d-4e2e-9637-0822f0f75522 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:58.718870 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826b9ba-26d4-470b-98b4-6fe7353928f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:58.719704 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826b9ba-26d4-470b-98b4-6fe7353928f0 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:58.719992 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826b9ba-26d4-470b-98b4-6fe7353928f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:58.720278 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826b9ba-26d4-470b-98b4-6fe7353928f0 None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:58.721473 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826b9ba-26d4-470b-98b4-6fe7353928f0 None admin] RBAC: Authorizing `identity:get_project(project_id=6525e5222e3342a3a26a6369518c7577)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:58.722940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826b9ba-26d4-470b-98b4-6fe7353928f0 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:58.730067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 164/317] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:22:58 2022] GET /identity/v3/projects/6525e5222e3342a3a26a6369518c7577 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:58.731445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6886b13-a751-435d-988b-a518d93f8aae None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:58.832850 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6886b13-a751-435d-988b-a518d93f8aae None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:58.860417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:58.870759 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45a675-b18a-4dcc-9f31-0b56d29f0558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:58.871753 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45a675-b18a-4dcc-9f31-0b56d29f0558 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:58.872281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45a675-b18a-4dcc-9f31-0b56d29f0558 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:58.872758 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45a675-b18a-4dcc-9f31-0b56d29f0558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:58.873787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e45a675-b18a-4dcc-9f31-0b56d29f0558 None admin] RBAC: Adding query filter params (role.id=ff3357541f744b6094e8e5fb5a79aa32, scope.project.id=6525e5222e3342a3a26a6369518c7577, user.id=cd8f2d3883334117899f753bede9c227) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:58.874290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e45a675-b18a-4dcc-9f31-0b56d29f0558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:58.875950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e45a675-b18a-4dcc-9f31-0b56d29f0558 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:58.882457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 154/318] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:22:58 2022] GET /identity/v3/role_assignments?user.id=cd8f2d3883334117899f753bede9c227&scope.project.id=6525e5222e3342a3a26a6369518c7577&role.id=ff3357541f744b6094e8e5fb5a79aa32 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:58.908695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6356ac-1263-4d3c-a23d-da0946912605 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:58.909052 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6356ac-1263-4d3c-a23d-da0946912605 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:58.909278 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6356ac-1263-4d3c-a23d-da0946912605 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:58.911100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 165/319] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:58.917881 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdbbaf-431e-46c4-99fe-ec4b590fdb55 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:58.918185 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdbbaf-431e-46c4-99fe-ec4b590fdb55 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:58.918458 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdbbaf-431e-46c4-99fe-ec4b590fdb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:58.936946 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8ffdbbaf-431e-46c4-99fe-ec4b590fdb55 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:58.938187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8ffdbbaf-431e-46c4-99fe-ec4b590fdb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:59.019306 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 155/320] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:22:59.044258 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6356ac-1263-4d3c-a23d-da0946912605 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:59.123116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:59.159746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6356ac-1263-4d3c-a23d-da0946912605 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:59.195540 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf149f2-f22c-48ec-a5ba-f5cdd07c40cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:59.196410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf149f2-f22c-48ec-a5ba-f5cdd07c40cb None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:59.196703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf149f2-f22c-48ec-a5ba-f5cdd07c40cb None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:59.196985 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf149f2-f22c-48ec-a5ba-f5cdd07c40cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:59.198171 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf149f2-f22c-48ec-a5ba-f5cdd07c40cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:59.199675 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf149f2-f22c-48ec-a5ba-f5cdd07c40cb None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:59.201303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 166/321] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:22:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:59.206942 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdbbaf-431e-46c4-99fe-ec4b590fdb55 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:59.311398 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdbbaf-431e-46c4-99fe-ec4b590fdb55 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:59.331418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:59.347927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b3a0e-5d4b-4652-9ecb-68b2100d2dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:59.348586 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b3a0e-5d4b-4652-9ecb-68b2100d2dbb None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:59.348882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b3a0e-5d4b-4652-9ecb-68b2100d2dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:59.349166 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b3a0e-5d4b-4652-9ecb-68b2100d2dbb None admin] PATH_INFO: `/v3/groups` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:59.349872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b3a0e-5d4b-4652-9ecb-68b2100d2dbb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:59.351405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b3a0e-5d4b-4652-9ecb-68b2100d2dbb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:59.366563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 156/322] 10.222.0.43 () {60 vars in 1134 bytes} [Thu Jun 23 22:22:59 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:22:59.382512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fd52f-620d-45e5-8c5e-6e4601cac49a None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:59.383135 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fd52f-620d-45e5-8c5e-6e4601cac49a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:59.383135 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fd52f-620d-45e5-8c5e-6e4601cac49a None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:59.384697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 167/323] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:59.390276 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17d9ce-66f6-423d-bf19-d65c176eacc0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:59.390593 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17d9ce-66f6-423d-bf19-d65c176eacc0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:59.390843 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17d9ce-66f6-423d-bf19-d65c176eacc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:59.408912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4e17d9ce-66f6-423d-bf19-d65c176eacc0 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:59.410108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4e17d9ce-66f6-423d-bf19-d65c176eacc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:59.484661 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 157/324] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:22:59.509474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fd52f-620d-45e5-8c5e-6e4601cac49a None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:59.523103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a761ca-68e4-4a28-a54c-6667b0e61c54 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:59.523478 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a761ca-68e4-4a28-a54c-6667b0e61c54 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:59.523763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a761ca-68e4-4a28-a54c-6667b0e61c54 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:59.525141 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 158/325] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:22:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:22:59.530161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a92a5-68dd-40f9-8ecd-20e2b9ad28c0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:59.530459 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a92a5-68dd-40f9-8ecd-20e2b9ad28c0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:59.531000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a92a5-68dd-40f9-8ecd-20e2b9ad28c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:59.547858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9a5a92a5-68dd-40f9-8ecd-20e2b9ad28c0 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:22:59.549055 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9a5a92a5-68dd-40f9-8ecd-20e2b9ad28c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:22:59.629845 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 159/326] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:22:59.639964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fd52f-620d-45e5-8c5e-6e4601cac49a None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:59.654980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a92a5-68dd-40f9-8ecd-20e2b9ad28c0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:59.677347 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073cc3d-5729-4c86-b5f9-3950c188246e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:59.678152 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073cc3d-5729-4c86-b5f9-3950c188246e None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:59.678491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073cc3d-5729-4c86-b5f9-3950c188246e None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:59.678818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d073cc3d-5729-4c86-b5f9-3950c188246e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:59.687090 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:59.695414 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d073cc3d-5729-4c86-b5f9-3950c188246e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:59.703076 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:22:59.704192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-d073cc3d-5729-4c86-b5f9-3950c188246e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:59.704192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:59.704192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:59.704192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:59.704192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:59.704192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:59.704192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:59.704192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:59.704192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:59.704192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:59.704192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:59.704192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:59.704192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:59.704192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:59.704192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:59.704192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:59.705128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:59.705128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:59.705128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:59.705128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:59.705128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:59.705128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:59.705128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:59.705128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:59.705128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:59.705128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:59.705128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:59.705128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:59.705128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:59.705128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:59.705128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:59.705128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:59.706033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:59.706033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:59.706033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:59.706033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:59.706033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:59.706033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:59.706033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:59.706033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:59.706033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:59.706033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:59.706033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:22:59.706033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:22:59.706033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 168/327] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:22:59.711595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073cc3d-5729-4c86-b5f9-3950c188246e None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:59.773796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a92a5-68dd-40f9-8ecd-20e2b9ad28c0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:59.819682 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132dfe2-d2cf-4daf-a354-f569b432d624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:59.820525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132dfe2-d2cf-4daf-a354-f569b432d624 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:59.820811 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132dfe2-d2cf-4daf-a354-f569b432d624 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:59.821160 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132dfe2-d2cf-4daf-a354-f569b432d624 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:59.837203 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5132dfe2-d2cf-4daf-a354-f569b432d624 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:22:59.845011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5132dfe2-d2cf-4daf-a354-f569b432d624 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:22:59.845011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:22:59.845011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:22:59.845011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:22:59.845011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:22:59.845011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:22:59.845011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:22:59.845011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:22:59.845011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:22:59.845011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:22:59.845011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:22:59.845011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:22:59.845011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:22:59.845011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:22:59.845011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:22:59.845011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:22:59.845961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:22:59.845961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:22:59.845961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:22:59.845961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:22:59.845961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:22:59.845961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:22:59.845961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:22:59.845961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:22:59.845961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:22:59.845961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:22:59.845961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:22:59.845961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:22:59.845961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:22:59.845961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:22:59.845961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:22:59.845961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:22:59.846939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:22:59.846939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:22:59.846939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:22:59.846939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:22:59.846939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:22:59.846939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:22:59.846939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:22:59.846939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:22:59.846939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:22:59.846939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:22:59.846939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:22:59.846939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:22:59.846939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 160/328] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:22:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:22:59.850336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d073cc3d-5729-4c86-b5f9-3950c188246e None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:22:59.858401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132dfe2-d2cf-4daf-a354-f569b432d624 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:59.894018 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c21d92-102a-4b03-950e-e123d6699f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:22:59.894623 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c21d92-102a-4b03-950e-e123d6699f48 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:22:59.896369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c21d92-102a-4b03-950e-e123d6699f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:22:59.896369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c21d92-102a-4b03-950e-e123d6699f48 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:22:59.896369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c21d92-102a-4b03-950e-e123d6699f48 None admin] RBAC: Adding query filter params (name=member) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:22:59.896369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c21d92-102a-4b03-950e-e123d6699f48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:22:59.898943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c21d92-102a-4b03-950e-e123d6699f48 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:22:59.911041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 169/329] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:22:59.917022 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c21d92-102a-4b03-950e-e123d6699f48 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:22:59.997726 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:00.001687 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132dfe2-d2cf-4daf-a354-f569b432d624 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:00.027282 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:00.047954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce87595-f40b-4e8a-a0a4-63b59821e726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:00.048382 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c21d92-102a-4b03-950e-e123d6699f48 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:00.048634 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce87595-f40b-4e8a-a0a4-63b59821e726 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:00.048921 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce87595-f40b-4e8a-a0a4-63b59821e726 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:00.049201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce87595-f40b-4e8a-a0a4-63b59821e726 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:00.050127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce87595-f40b-4e8a-a0a4-63b59821e726 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:00.050442 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce87595-f40b-4e8a-a0a4-63b59821e726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:00.053929 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce87595-f40b-4e8a-a0a4-63b59821e726 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:00.064517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 161/330] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:22:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:00.070690 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce87595-f40b-4e8a-a0a4-63b59821e726 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:00.092719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38be2c-02a6-45ea-87c1-f5b5ffe25a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:00.093607 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38be2c-02a6-45ea-87c1-f5b5ffe25a9c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:00.093937 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38be2c-02a6-45ea-87c1-f5b5ffe25a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:00.094268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38be2c-02a6-45ea-87c1-f5b5ffe25a9c None admin] PATH_INFO: `/v3/users/dc76f6fb1e774047827d88badcebc53a` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:00.095607 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38be2c-02a6-45ea-87c1-f5b5ffe25a9c None admin] RBAC: Authorizing `identity:get_user(user_id=dc76f6fb1e774047827d88badcebc53a)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:00.097079 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38be2c-02a6-45ea-87c1-f5b5ffe25a9c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:00.099779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 170/331] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:22:59 2022] GET /identity/v3/users/dc76f6fb1e774047827d88badcebc53a => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:00.106783 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38be2c-02a6-45ea-87c1-f5b5ffe25a9c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:00.188525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce87595-f40b-4e8a-a0a4-63b59821e726 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:00.226613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c42043-5366-42f7-9c82-b4c01cb2cc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:00.227532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c42043-5366-42f7-9c82-b4c01cb2cc21 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:00.227818 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c42043-5366-42f7-9c82-b4c01cb2cc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:00.228097 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c42043-5366-42f7-9c82-b4c01cb2cc21 None admin] PATH_INFO: `/v3/users/41aff0b978824cd389a2da81b4490e52` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:00.229320 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c42043-5366-42f7-9c82-b4c01cb2cc21 None admin] RBAC: Authorizing `identity:get_user(user_id=41aff0b978824cd389a2da81b4490e52)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:00.230763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c42043-5366-42f7-9c82-b4c01cb2cc21 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:00.232742 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 162/332] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:23:00 2022] GET /identity/v3/users/41aff0b978824cd389a2da81b4490e52 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:00.238297 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c42043-5366-42f7-9c82-b4c01cb2cc21 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:00.252662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38be2c-02a6-45ea-87c1-f5b5ffe25a9c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:00.303285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:00.315281 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:00.328385 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2364c09-9246-4cf7-b94a-8b1d5a89961a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:00.329053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2364c09-9246-4cf7-b94a-8b1d5a89961a None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:00.329341 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2364c09-9246-4cf7-b94a-8b1d5a89961a None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:00.329623 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2364c09-9246-4cf7-b94a-8b1d5a89961a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:00.330458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2364c09-9246-4cf7-b94a-8b1d5a89961a None admin] RBAC: Adding query filter params (role.id=1baba2239751440eb25af0d0fd2d072d, user.id=dc76f6fb1e774047827d88badcebc53a, scope.system=all) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:00.330782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2364c09-9246-4cf7-b94a-8b1d5a89961a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:00.332253 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2364c09-9246-4cf7-b94a-8b1d5a89961a None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:00.361449 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 171/333] 10.222.0.43 () {56 vars in 1308 bytes} [Thu Jun 23 22:23:00 2022] GET /identity/v3/role_assignments?user.id=dc76f6fb1e774047827d88badcebc53a&scope.system=all&role.id=1baba2239751440eb25af0d0fd2d072d => generated 510 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:00.408180 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c42043-5366-42f7-9c82-b4c01cb2cc21 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:00.445847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8df19-1757-406f-b300-2203d65c4c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:00.446629 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8df19-1757-406f-b300-2203d65c4c01 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:00.446912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8df19-1757-406f-b300-2203d65c4c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:00.447216 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8df19-1757-406f-b300-2203d65c4c01 None admin] PATH_INFO: `/v3/system/users/41aff0b978824cd389a2da81b4490e52/roles/ff3357541f744b6094e8e5fb5a79aa32` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:00.448706 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8df19-1757-406f-b300-2203d65c4c01 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=41aff0b978824cd389a2da81b4490e52, role_id=ff3357541f744b6094e8e5fb5a79aa32)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:00.450081 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8df19-1757-406f-b300-2203d65c4c01 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:00.462576 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 163/334] 10.222.0.43 () {58 vars in 1334 bytes} [Thu Jun 23 22:23:00 2022] PUT /identity/v3/system/users/41aff0b978824cd389a2da81b4490e52/roles/ff3357541f744b6094e8e5fb5a79aa32 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:23:01.706698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8a962-6aa9-433c-bda1-9d0f8b9ccbf9 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:01.707245 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8a962-6aa9-433c-bda1-9d0f8b9ccbf9 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:01.707350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8a962-6aa9-433c-bda1-9d0f8b9ccbf9 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:01.708724 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 172/335] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:01.714879 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83fd9d-5524-4a0a-9c49-4b8e4c114c85 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:01.715350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83fd9d-5524-4a0a-9c49-4b8e4c114c85 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:01.715501 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83fd9d-5524-4a0a-9c49-4b8e4c114c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:01.735901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7c83fd9d-5524-4a0a-9c49-4b8e4c114c85 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:01.737087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7c83fd9d-5524-4a0a-9c49-4b8e4c114c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:01.818382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 164/336] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:01.866340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8a962-6aa9-433c-bda1-9d0f8b9ccbf9 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:01.979865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8a962-6aa9-433c-bda1-9d0f8b9ccbf9 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:02.017681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd76b4-d8e9-435f-a64b-1359acf713b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:02.018487 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd76b4-d8e9-435f-a64b-1359acf713b4 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:02.018775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd76b4-d8e9-435f-a64b-1359acf713b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:02.019120 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd76b4-d8e9-435f-a64b-1359acf713b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:02.031503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabd76b4-d8e9-435f-a64b-1359acf713b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:02.040217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-aabd76b4-d8e9-435f-a64b-1359acf713b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:23:02.040217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:02.040217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:02.040217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:02.040217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:02.040217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:02.040217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:02.040217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:02.040217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:02.040217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:02.040217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:02.040217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:02.040217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:02.040217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:02.040217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:02.040217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:02.041107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:02.041107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:02.041107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:02.041107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:02.041107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:02.041107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:02.041107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:02.041107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:02.041107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:02.041107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:02.041107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:02.041107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:02.041107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:02.041107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:02.041107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:02.041107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:02.041989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:02.041989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:02.041989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:02.041989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:02.041989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:02.041989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:02.041989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:02.041989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:02.041989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:02.041989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:02.041989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:23:02.041989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:02.041989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 173/337] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:02.049060 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83fd9d-5524-4a0a-9c49-4b8e4c114c85 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:02.090472 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:02.215050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83fd9d-5524-4a0a-9c49-4b8e4c114c85 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:02.259513 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa339f8d-3122-434e-a68b-848096f03158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:02.260113 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa339f8d-3122-434e-a68b-848096f03158 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:02.260408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa339f8d-3122-434e-a68b-848096f03158 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:02.260698 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa339f8d-3122-434e-a68b-848096f03158 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:02.261466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa339f8d-3122-434e-a68b-848096f03158 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:02.261786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa339f8d-3122-434e-a68b-848096f03158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:02.267287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa339f8d-3122-434e-a68b-848096f03158 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:02.277389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 165/338] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:23:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:02.282468 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd76b4-d8e9-435f-a64b-1359acf713b4 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:02.345660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:02.355218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f8670-5a75-448f-a747-9a146d7740c6 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:02.355882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f8670-5a75-448f-a747-9a146d7740c6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:02.356369 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f8670-5a75-448f-a747-9a146d7740c6 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:02.359003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 166/339] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:02.363237 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60805dca-01b1-43ef-9780-43ab28d56028 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:02.363803 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60805dca-01b1-43ef-9780-43ab28d56028 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:02.364292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60805dca-01b1-43ef-9780-43ab28d56028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:02.394369 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-60805dca-01b1-43ef-9780-43ab28d56028 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:02.395959 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-60805dca-01b1-43ef-9780-43ab28d56028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:02.423463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd76b4-d8e9-435f-a64b-1359acf713b4 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:02.496033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42844c5-ee7a-43dd-8102-db93e57cec77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:02.496662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42844c5-ee7a-43dd-8102-db93e57cec77 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:02.497133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42844c5-ee7a-43dd-8102-db93e57cec77 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:02.497427 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42844c5-ee7a-43dd-8102-db93e57cec77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:02.549999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 167/340] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:02.577901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60805dca-01b1-43ef-9780-43ab28d56028 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:02.609135 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:02.611618 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42844c5-ee7a-43dd-8102-db93e57cec77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:02.614944 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42844c5-ee7a-43dd-8102-db93e57cec77 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:02.718842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60805dca-01b1-43ef-9780-43ab28d56028 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:02.721180 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:02.774541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64401d7b-f3d9-48b6-bcc1-d0a893926ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:02.775729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64401d7b-f3d9-48b6-bcc1-d0a893926ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:02.776241 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64401d7b-f3d9-48b6-bcc1-d0a893926ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:02.776721 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64401d7b-f3d9-48b6-bcc1-d0a893926ab6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:02.791599 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64401d7b-f3d9-48b6-bcc1-d0a893926ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:02.800108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-64401d7b-f3d9-48b6-bcc1-d0a893926ab6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:23:02.800108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:02.800108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:02.800108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:02.800108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:02.800108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:02.800108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:02.800108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:02.800108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:02.800108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:02.800108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:02.800108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:02.800108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:02.800108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:02.800108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:02.800108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:02.801019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:02.801019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:02.801019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:02.801019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:02.801019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:02.801019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:02.801019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:02.801019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:02.801019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:02.801019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:02.801019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:02.801019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:02.801019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:02.801019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:02.801019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:02.801019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:02.801819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:02.801819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:02.801819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:02.801819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:02.801819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:02.801819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:02.801819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:02.801819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:02.801819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:02.801819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:02.801819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:23:02.801819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:02.804867 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 168/341] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:23:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:02.810220 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64401d7b-f3d9-48b6-bcc1-d0a893926ab6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:02.836175 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 174/342] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:02.846948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42844c5-ee7a-43dd-8102-db93e57cec77 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:02.924795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:02.952917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64401d7b-f3d9-48b6-bcc1-d0a893926ab6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:03.014032 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9f506-3cfa-47f7-bd0d-5c21522aa734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:03.015140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9f506-3cfa-47f7-bd0d-5c21522aa734 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:03.018270 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9f506-3cfa-47f7-bd0d-5c21522aa734 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:03.018772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9f506-3cfa-47f7-bd0d-5c21522aa734 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:03.019772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9f506-3cfa-47f7-bd0d-5c21522aa734 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:03.020287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9f506-3cfa-47f7-bd0d-5c21522aa734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:03.021887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9f506-3cfa-47f7-bd0d-5c21522aa734 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:03.044695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 169/343] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:03.050182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9f506-3cfa-47f7-bd0d-5c21522aa734 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:03.119856 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42844c5-ee7a-43dd-8102-db93e57cec77 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:03.189647 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460715b-3602-493e-977c-412eaff35263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:03.190527 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460715b-3602-493e-977c-412eaff35263 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:03.190815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460715b-3602-493e-977c-412eaff35263 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:03.191123 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460715b-3602-493e-977c-412eaff35263 None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:03.193525 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460715b-3602-493e-977c-412eaff35263 None admin] RBAC: Authorizing `identity:get_project(project_id=53390afb4a404df79ab2d9ae31e8d058)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:03.195175 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460715b-3602-493e-977c-412eaff35263 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:03.199108 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 175/344] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:23:02 2022] GET /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058 => generated 298 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:03.200404 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3294d0a-1cb3-4223-8453-5ec760023470 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:03.200699 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3294d0a-1cb3-4223-8453-5ec760023470 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:03.201057 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3294d0a-1cb3-4223-8453-5ec760023470 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:03.202608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 176/345] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:03.203271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3294d0a-1cb3-4223-8453-5ec760023470 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:03.228769 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9f506-3cfa-47f7-bd0d-5c21522aa734 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:03.248696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:03.277526 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb213e7-4cda-4f23-bad4-5c25017790a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:03.278182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb213e7-4cda-4f23-bad4-5c25017790a3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:03.278466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb213e7-4cda-4f23-bad4-5c25017790a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:03.278745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb213e7-4cda-4f23-bad4-5c25017790a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:03.291597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:03.372670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb213e7-4cda-4f23-bad4-5c25017790a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:03.374034 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb213e7-4cda-4f23-bad4-5c25017790a3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:03.480221 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3294d0a-1cb3-4223-8453-5ec760023470 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:03.525323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 170/346] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:03.527509 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294e457-6387-4737-a9d0-4091937b0b17 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:03.527805 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294e457-6387-4737-a9d0-4091937b0b17 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:03.528088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294e457-6387-4737-a9d0-4091937b0b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:03.548203 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1294e457-6387-4737-a9d0-4091937b0b17 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:03.548521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159b925-698c-45d8-8c5b-d423d30e63b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:03.549322 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159b925-698c-45d8-8c5b-d423d30e63b2 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:03.549424 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1294e457-6387-4737-a9d0-4091937b0b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:03.549617 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159b925-698c-45d8-8c5b-d423d30e63b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:03.549906 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159b925-698c-45d8-8c5b-d423d30e63b2 None admin] PATH_INFO: `/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:03.559582 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159b925-698c-45d8-8c5b-d423d30e63b2 None admin] RBAC: Authorizing `identity:get_group(group_id=53f45b0a49154be9a2c7124b7ee49fa7)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:03.561180 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159b925-698c-45d8-8c5b-d423d30e63b2 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:03.563199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 177/347] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 22:23:03 2022] GET /identity/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7 => generated 226 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:03.567841 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159b925-698c-45d8-8c5b-d423d30e63b2 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:03.592815 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:03.655024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 171/348] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:03.655024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294e457-6387-4737-a9d0-4091937b0b17 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:03.710950 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:03.744205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159b925-698c-45d8-8c5b-d423d30e63b2 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:03.758457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294e457-6387-4737-a9d0-4091937b0b17 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:03.794168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcfdd6-0cfb-4bd3-9b6d-cba74b6668fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:03.795581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcfdd6-0cfb-4bd3-9b6d-cba74b6668fe None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:03.796139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcfdd6-0cfb-4bd3-9b6d-cba74b6668fe None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:03.796469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcfdd6-0cfb-4bd3-9b6d-cba74b6668fe None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:03.797893 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdcfdd6-0cfb-4bd3-9b6d-cba74b6668fe None admin] RBAC: Authorizing `identity:get_project(project_id=53390afb4a404df79ab2d9ae31e8d058)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:03.799580 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdcfdd6-0cfb-4bd3-9b6d-cba74b6668fe None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:03.800789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccb7c0-e4fb-4260-bbef-663335afe85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:03.801292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 178/349] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:23:03 2022] GET /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:03.801787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccb7c0-e4fb-4260-bbef-663335afe85b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:03.802054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcfdd6-0cfb-4bd3-9b6d-cba74b6668fe None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:03.802164 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccb7c0-e4fb-4260-bbef-663335afe85b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:03.802345 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccb7c0-e4fb-4260-bbef-663335afe85b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:03.803283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdccb7c0-e4fb-4260-bbef-663335afe85b None admin] RBAC: Adding query filter params (group.id=53f45b0a49154be9a2c7124b7ee49fa7, role.id=1baba2239751440eb25af0d0fd2d072d, scope.project.id=53390afb4a404df79ab2d9ae31e8d058) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:03.804179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdccb7c0-e4fb-4260-bbef-663335afe85b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:03.805636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdccb7c0-e4fb-4260-bbef-663335afe85b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:03.821279 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 172/350] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 22:23:03 2022] GET /identity/v3/role_assignments?group.id=53f45b0a49154be9a2c7124b7ee49fa7&scope.project.id=53390afb4a404df79ab2d9ae31e8d058&role.id=1baba2239751440eb25af0d0fd2d072d => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:03.822759 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dec3d-eef0-4df9-a0e8-4c6d44f5be50 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:03.823121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dec3d-eef0-4df9-a0e8-4c6d44f5be50 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:03.823389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0dec3d-eef0-4df9-a0e8-4c6d44f5be50 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:03.824843 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 173/351] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:03.825633 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dec3d-eef0-4df9-a0e8-4c6d44f5be50 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:03.864683 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:03.996412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0dec3d-eef0-4df9-a0e8-4c6d44f5be50 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:03.997527 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcfdd6-0cfb-4bd3-9b6d-cba74b6668fe None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:04.044382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c24f1-d5ac-4e4c-a284-ea0f06861102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:04.044611 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879553c4-873c-417d-86e3-ca62dc7817bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:04.044611 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879553c4-873c-417d-86e3-ca62dc7817bd None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:04.044784 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c24f1-d5ac-4e4c-a284-ea0f06861102 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:04.044858 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879553c4-873c-417d-86e3-ca62dc7817bd None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:04.044929 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c24f1-d5ac-4e4c-a284-ea0f06861102 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:04.045000 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879553c4-873c-417d-86e3-ca62dc7817bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:04.045147 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c24f1-d5ac-4e4c-a284-ea0f06861102 None admin] PATH_INFO: `/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:04.046173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c24f1-d5ac-4e4c-a284-ea0f06861102 None admin] RBAC: Authorizing `identity:get_group(group_id=53f45b0a49154be9a2c7124b7ee49fa7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:04.047644 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c24f1-d5ac-4e4c-a284-ea0f06861102 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:04.049269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 174/352] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 22:23:03 2022] GET /identity/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:04.050430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922cbd5-97c0-44dd-bb78-ca920964577d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:04.050846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922cbd5-97c0-44dd-bb78-ca920964577d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:04.051683 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922cbd5-97c0-44dd-bb78-ca920964577d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:04.057738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879553c4-873c-417d-86e3-ca62dc7817bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:04.069639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-879553c4-873c-417d-86e3-ca62dc7817bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:23:04.069639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:04.069639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:04.069639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:04.069639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:04.069639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:04.069639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:04.069639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:04.069639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:04.069639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:04.069639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:04.069639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:04.069639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:04.069639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:04.069639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:04.069639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:04.070635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:04.070635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:04.070635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:04.070635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:04.070635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:04.070635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:04.070635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:04.070635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:04.070635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:04.070635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:04.070635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:04.070635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:04.070635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:04.070635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:04.070635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:04.070635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:04.071522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:04.071522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:04.071522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:04.071522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:04.071522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:04.071522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:04.071522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:04.071522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:04.071522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:04.071522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:04.071522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:23:04.071522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:04.073979 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 179/353] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:04.075558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b0e50-59ed-4a02-95f2-a9d6429fb7fe None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:04.076080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8922cbd5-97c0-44dd-bb78-ca920964577d None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:04.076206 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b0e50-59ed-4a02-95f2-a9d6429fb7fe None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:04.076677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b0e50-59ed-4a02-95f2-a9d6429fb7fe None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:04.077276 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8922cbd5-97c0-44dd-bb78-ca920964577d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:04.078654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 180/354] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:04.083500 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b0e50-59ed-4a02-95f2-a9d6429fb7fe None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:04.094007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:04.177021 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 175/355] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:04.178117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922cbd5-97c0-44dd-bb78-ca920964577d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:04.216601 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:04.245672 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b0e50-59ed-4a02-95f2-a9d6429fb7fe None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:04.292044 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b16fa-7240-4511-90ef-2ab054152543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:04.292840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b16fa-7240-4511-90ef-2ab054152543 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:04.293137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b16fa-7240-4511-90ef-2ab054152543 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:04.293421 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b16fa-7240-4511-90ef-2ab054152543 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:04.295807 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b16fa-7240-4511-90ef-2ab054152543 None admin] RBAC: Adding query filter params (group.id=53f45b0a49154be9a2c7124b7ee49fa7, role.id=61397d4a1ccd4a8ca479f98186ed489b, scope.project.id=53390afb4a404df79ab2d9ae31e8d058) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:04.295807 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b16fa-7240-4511-90ef-2ab054152543 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:04.296001 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b16fa-7240-4511-90ef-2ab054152543 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:04.304591 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 181/356] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 22:23:04 2022] GET /identity/v3/role_assignments?group.id=53f45b0a49154be9a2c7124b7ee49fa7&scope.project.id=53390afb4a404df79ab2d9ae31e8d058&role.id=61397d4a1ccd4a8ca479f98186ed489b => generated 260 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:04.311001 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e1cc3-38ec-4e54-a9d9-fbee2cf99cab None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:04.311001 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e1cc3-38ec-4e54-a9d9-fbee2cf99cab None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:04.311001 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e1cc3-38ec-4e54-a9d9-fbee2cf99cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:04.312924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922cbd5-97c0-44dd-bb78-ca920964577d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:04.327237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-885e1cc3-38ec-4e54-a9d9-fbee2cf99cab None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:04.328462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-885e1cc3-38ec-4e54-a9d9-fbee2cf99cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:04.376623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdad90-00ea-45dd-9945-d5d159cf5f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:04.377974 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdad90-00ea-45dd-9945-d5d159cf5f8c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:04.377974 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdad90-00ea-45dd-9945-d5d159cf5f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:04.377974 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdad90-00ea-45dd-9945-d5d159cf5f8c None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:04.378611 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdad90-00ea-45dd-9945-d5d159cf5f8c None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:04.378960 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdad90-00ea-45dd-9945-d5d159cf5f8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:04.380333 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdad90-00ea-45dd-9945-d5d159cf5f8c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:04.394456 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 176/357] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:23:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:04.395899 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdad90-00ea-45dd-9945-d5d159cf5f8c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:04.422167 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:04.429383 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 182/358] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:04.432472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44987c-349e-4bfb-a723-f184ff58c64e None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:04.433111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44987c-349e-4bfb-a723-f184ff58c64e None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:04.433884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44987c-349e-4bfb-a723-f184ff58c64e None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:04.440580 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 183/359] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:04.440580 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44987c-349e-4bfb-a723-f184ff58c64e None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:04.542739 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:04.556941 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdad90-00ea-45dd-9945-d5d159cf5f8c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:04.596956 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44987c-349e-4bfb-a723-f184ff58c64e None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:04.614115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7362196-01a6-4ea3-bbad-b08a73a9b015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:04.615515 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7362196-01a6-4ea3-bbad-b08a73a9b015 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:04.616493 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7362196-01a6-4ea3-bbad-b08a73a9b015 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:04.617520 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7362196-01a6-4ea3-bbad-b08a73a9b015 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:04.632956 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7362196-01a6-4ea3-bbad-b08a73a9b015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:04.637235 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f517c2-50e9-44ed-956a-8650cea959c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:04.637966 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f517c2-50e9-44ed-956a-8650cea959c3 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:04.638343 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f517c2-50e9-44ed-956a-8650cea959c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:04.638697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f517c2-50e9-44ed-956a-8650cea959c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:04.643278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d7362196-01a6-4ea3-bbad-b08a73a9b015 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:23:04.643278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:04.643278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:04.643278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:04.643278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:04.643278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:04.643278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:04.643278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:04.643278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:04.643278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:04.643278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:04.643278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:04.643278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:04.643278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:04.643278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:04.643278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:04.644151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:04.644151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:04.644151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:04.644151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:04.644151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:04.644151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:04.644151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:04.644151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:04.644151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:04.644151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:04.644151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:04.644151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:04.644151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:04.644151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:04.644151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:04.644151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:04.644950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:04.644950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:04.644950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:04.644950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:04.644950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:04.644950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:04.644950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:04.644950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:04.644950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:04.644950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:04.644950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:23:04.644950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:04.646524 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 177/360] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:23:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:04.650989 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba95f00-e3bd-44d6-9dde-7792bf2e58b6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:04.650989 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba95f00-e3bd-44d6-9dde-7792bf2e58b6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:04.650989 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba95f00-e3bd-44d6-9dde-7792bf2e58b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:04.672805 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fba95f00-e3bd-44d6-9dde-7792bf2e58b6 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:04.673993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fba95f00-e3bd-44d6-9dde-7792bf2e58b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:04.735347 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:04.738481 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f517c2-50e9-44ed-956a-8650cea959c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:04.740010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f517c2-50e9-44ed-956a-8650cea959c3 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:04.767793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 178/361] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:04.770198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba95f00-e3bd-44d6-9dde-7792bf2e58b6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:04.875599 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 184/362] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:04.876892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:04.878602 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f517c2-50e9-44ed-956a-8650cea959c3 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:04.900528 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba95f00-e3bd-44d6-9dde-7792bf2e58b6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:04.947958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b919a8c-947b-4d29-a8de-a1801a1bc7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:04.948773 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b919a8c-947b-4d29-a8de-a1801a1bc7f2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:04.949058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b919a8c-947b-4d29-a8de-a1801a1bc7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:04.949338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b919a8c-947b-4d29-a8de-a1801a1bc7f2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:04.961390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b919a8c-947b-4d29-a8de-a1801a1bc7f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:04.969736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3b919a8c-947b-4d29-a8de-a1801a1bc7f2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:23:04.969736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:04.969736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:04.969736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:04.969736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:04.969736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:04.969736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:04.969736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:04.969736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:04.969736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:04.969736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:04.969736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:04.969736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:04.969736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:04.969736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:04.969736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:04.970655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:04.970655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:04.970655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:04.970655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:04.970655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:04.970655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:04.970655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:04.970655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:04.970655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:04.970655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:04.970655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:04.970655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:04.970655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:04.970655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:04.970655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:04.970655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:04.971561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:04.971561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:04.971561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:04.971561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:04.971561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:04.971561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:04.971561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:04.971561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:04.971561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:04.971561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:04.971561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 22:23:04.971561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:04.971561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 179/363] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:04.972269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b919a8c-947b-4d29-a8de-a1801a1bc7f2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:05.002012 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:05.006106 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f517c2-50e9-44ed-956a-8650cea959c3 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:05.050059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa36d07-5d65-4b48-94a5-b37a3611e0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:05.051145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa36d07-5d65-4b48-94a5-b37a3611e0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:05.051680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa36d07-5d65-4b48-94a5-b37a3611e0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:05.052165 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa36d07-5d65-4b48-94a5-b37a3611e0c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:05.053130 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa36d07-5d65-4b48-94a5-b37a3611e0c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:05.054326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa36d07-5d65-4b48-94a5-b37a3611e0c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:05.056605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa36d07-5d65-4b48-94a5-b37a3611e0c1 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:05.069130 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 185/364] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:05.070273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa36d07-5d65-4b48-94a5-b37a3611e0c1 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:05.093390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b919a8c-947b-4d29-a8de-a1801a1bc7f2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:05.132176 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f1967-0422-4003-9c1d-60429854f88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:05.133117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f1967-0422-4003-9c1d-60429854f88d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:05.133498 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f1967-0422-4003-9c1d-60429854f88d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:05.133869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f1967-0422-4003-9c1d-60429854f88d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:05.145947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f1967-0422-4003-9c1d-60429854f88d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:05.155105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4e8f1967-0422-4003-9c1d-60429854f88d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:23:05.155105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:05.155105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:05.155105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:05.155105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:05.155105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:05.155105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:05.155105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:05.155105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:05.155105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:05.155105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:05.155105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:05.155105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:05.155105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:05.155105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:05.155105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:05.156142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:05.156142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:05.156142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:05.156142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:05.156142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:05.156142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:05.156142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:05.156142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:05.156142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:05.156142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:05.156142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:05.156142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:05.156142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:05.156142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:05.156142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:05.156142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:05.156949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:05.156949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:05.156949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:05.156949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:05.156949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:05.156949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:05.156949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:05.156949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:05.156949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:05.156949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:05.156949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:23:05.156949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:05.156949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 180/365] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:23:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:05.158119 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f1967-0422-4003-9c1d-60429854f88d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:05.162055 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:05.197348 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa36d07-5d65-4b48-94a5-b37a3611e0c1 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:05.236317 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d363e73-f4c8-4c3c-9943-ac30b7788626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:05.237467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d363e73-f4c8-4c3c-9943-ac30b7788626 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:05.237939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d363e73-f4c8-4c3c-9943-ac30b7788626 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:05.238397 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d363e73-f4c8-4c3c-9943-ac30b7788626 None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:05.239834 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d363e73-f4c8-4c3c-9943-ac30b7788626 None admin] RBAC: Authorizing `identity:get_project(project_id=6525e5222e3342a3a26a6369518c7577)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:05.242575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d363e73-f4c8-4c3c-9943-ac30b7788626 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:05.244628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 186/366] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:23:05 2022] GET /identity/v3/projects/6525e5222e3342a3a26a6369518c7577 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:05.245621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d363e73-f4c8-4c3c-9943-ac30b7788626 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:05.277051 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:05.278016 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f1967-0422-4003-9c1d-60429854f88d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:05.317437 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1f7ba-d39d-422c-8e53-0e627b983271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:05.318210 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1f7ba-d39d-422c-8e53-0e627b983271 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:05.318592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1f7ba-d39d-422c-8e53-0e627b983271 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:05.318980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1f7ba-d39d-422c-8e53-0e627b983271 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:05.319901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1f7ba-d39d-422c-8e53-0e627b983271 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:05.320278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1f7ba-d39d-422c-8e53-0e627b983271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:05.321840 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1f7ba-d39d-422c-8e53-0e627b983271 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:05.332649 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 181/367] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:23:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:05.338958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1f7ba-d39d-422c-8e53-0e627b983271 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:05.362612 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d363e73-f4c8-4c3c-9943-ac30b7788626 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:05.398843 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2463bf-e111-4e18-83c5-60a61bdebbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:05.399595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2463bf-e111-4e18-83c5-60a61bdebbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:05.399988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2463bf-e111-4e18-83c5-60a61bdebbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:05.400419 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2463bf-e111-4e18-83c5-60a61bdebbb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:05.410910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:05.455364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1f7ba-d39d-422c-8e53-0e627b983271 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:05.472551 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2463bf-e111-4e18-83c5-60a61bdebbb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:05.474020 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2463bf-e111-4e18-83c5-60a61bdebbb4 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:05.494851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03911044-1e48-4b09-991d-6963b2216d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:05.496019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03911044-1e48-4b09-991d-6963b2216d61 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:05.496504 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03911044-1e48-4b09-991d-6963b2216d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:05.497196 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03911044-1e48-4b09-991d-6963b2216d61 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:05.498149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03911044-1e48-4b09-991d-6963b2216d61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:05.498720 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03911044-1e48-4b09-991d-6963b2216d61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:05.500409 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03911044-1e48-4b09-991d-6963b2216d61 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:05.504977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:05.512377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 182/368] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:05.513459 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03911044-1e48-4b09-991d-6963b2216d61 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:05.579738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 187/369] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:05.580426 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2463bf-e111-4e18-83c5-60a61bdebbb4 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:05.625795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03911044-1e48-4b09-991d-6963b2216d61 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:05.657020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:05.669867 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52962d20-7b46-4714-87ab-a185c93718dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:05.671107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52962d20-7b46-4714-87ab-a185c93718dc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:05.671888 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52962d20-7b46-4714-87ab-a185c93718dc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:05.672912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52962d20-7b46-4714-87ab-a185c93718dc None admin] PATH_INFO: `/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:05.675143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52962d20-7b46-4714-87ab-a185c93718dc None admin] RBAC: Authorizing `identity:get_group(group_id=53f45b0a49154be9a2c7124b7ee49fa7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:05.676835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52962d20-7b46-4714-87ab-a185c93718dc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:05.679575 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 183/370] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 22:23:05 2022] GET /identity/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:05.681439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52962d20-7b46-4714-87ab-a185c93718dc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:05.702238 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2463bf-e111-4e18-83c5-60a61bdebbb4 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:05.736108 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:05.742571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34e260-c8da-47f4-9688-686ed26263ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:05.743979 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34e260-c8da-47f4-9688-686ed26263ed None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:05.744730 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34e260-c8da-47f4-9688-686ed26263ed None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:05.745355 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34e260-c8da-47f4-9688-686ed26263ed None admin] PATH_INFO: `/v3/users/41aff0b978824cd389a2da81b4490e52` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:05.746945 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff34e260-c8da-47f4-9688-686ed26263ed None admin] RBAC: Authorizing `identity:get_user(user_id=41aff0b978824cd389a2da81b4490e52)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:05.748490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff34e260-c8da-47f4-9688-686ed26263ed None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:05.750738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 188/371] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:23:05 2022] GET /identity/v3/users/41aff0b978824cd389a2da81b4490e52 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:05.751629 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34e260-c8da-47f4-9688-686ed26263ed None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:05.796468 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52962d20-7b46-4714-87ab-a185c93718dc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:05.837621 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b2b00-303c-4f37-8ef8-f2122e22a03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:05.838622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b2b00-303c-4f37-8ef8-f2122e22a03c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:05.839106 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b2b00-303c-4f37-8ef8-f2122e22a03c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:05.839590 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b2b00-303c-4f37-8ef8-f2122e22a03c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:05.870188 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34e260-c8da-47f4-9688-686ed26263ed None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:05.908809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdb189-8b67-4332-a0ef-b2946d0dfc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:05.909693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdb189-8b67-4332-a0ef-b2946d0dfc42 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:05.910060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdb189-8b67-4332-a0ef-b2946d0dfc42 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:05.910420 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdb189-8b67-4332-a0ef-b2946d0dfc42 None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:05.911877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cdb189-8b67-4332-a0ef-b2946d0dfc42 None admin] RBAC: Authorizing `identity:get_project(project_id=6525e5222e3342a3a26a6369518c7577)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:05.913560 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cdb189-8b67-4332-a0ef-b2946d0dfc42 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:05.915863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 189/372] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:23:05 2022] GET /identity/v3/projects/6525e5222e3342a3a26a6369518c7577 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:05.916718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdb189-8b67-4332-a0ef-b2946d0dfc42 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:05.927215 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:05.935974 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b2b00-303c-4f37-8ef8-f2122e22a03c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:05.937442 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b2b00-303c-4f37-8ef8-f2122e22a03c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:06.008842 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:06.056742 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdb189-8b67-4332-a0ef-b2946d0dfc42 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:06.061918 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 184/373] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:06.062832 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b2b00-303c-4f37-8ef8-f2122e22a03c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:06.103453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3f1b0-9aa2-42bc-8514-14772e33acf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:06.104128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3f1b0-9aa2-42bc-8514-14772e33acf8 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:06.104416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3f1b0-9aa2-42bc-8514-14772e33acf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:06.104906 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3f1b0-9aa2-42bc-8514-14772e33acf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:06.106557 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3f1b0-9aa2-42bc-8514-14772e33acf8 None admin] RBAC: Adding query filter params (group.id=53f45b0a49154be9a2c7124b7ee49fa7, role.id=1baba2239751440eb25af0d0fd2d072d, scope.project.id=6525e5222e3342a3a26a6369518c7577) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:06.106877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3f1b0-9aa2-42bc-8514-14772e33acf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:06.108345 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3f1b0-9aa2-42bc-8514-14772e33acf8 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:06.116047 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 190/374] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 22:23:05 2022] GET /identity/v3/role_assignments?group.id=53f45b0a49154be9a2c7124b7ee49fa7&scope.project.id=6525e5222e3342a3a26a6369518c7577&role.id=1baba2239751440eb25af0d0fd2d072d => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:06.116880 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3f1b0-9aa2-42bc-8514-14772e33acf8 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:06.203596 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:06.222430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b2b00-303c-4f37-8ef8-f2122e22a03c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:06.306575 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8196cb-1939-4ca5-9e25-4382fbea1c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:06.307540 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8196cb-1939-4ca5-9e25-4382fbea1c97 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:06.307827 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8196cb-1939-4ca5-9e25-4382fbea1c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:06.308103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8196cb-1939-4ca5-9e25-4382fbea1c97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:06.308954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8196cb-1939-4ca5-9e25-4382fbea1c97 None admin] RBAC: Adding query filter params (role.id=ff3357541f744b6094e8e5fb5a79aa32, user.id=41aff0b978824cd389a2da81b4490e52, scope.system=all) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:06.309274 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8196cb-1939-4ca5-9e25-4382fbea1c97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:06.310747 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8196cb-1939-4ca5-9e25-4382fbea1c97 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:06.338475 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3f1b0-9aa2-42bc-8514-14772e33acf8 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:06.342702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 185/375] 10.222.0.43 () {56 vars in 1308 bytes} [Thu Jun 23 22:23:06 2022] GET /identity/v3/role_assignments?user.id=41aff0b978824cd389a2da81b4490e52&scope.system=all&role.id=ff3357541f744b6094e8e5fb5a79aa32 => generated 510 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:06.344138 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8196cb-1939-4ca5-9e25-4382fbea1c97 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:06.376339 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:06.387476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005abe6c-8b4e-4c86-8a8a-e6e1f1c9e03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:06.388797 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005abe6c-8b4e-4c86-8a8a-e6e1f1c9e03f None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:06.389087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005abe6c-8b4e-4c86-8a8a-e6e1f1c9e03f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:06.389367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005abe6c-8b4e-4c86-8a8a-e6e1f1c9e03f None admin] PATH_INFO: `/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:06.390731 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005abe6c-8b4e-4c86-8a8a-e6e1f1c9e03f None admin] RBAC: Authorizing `identity:get_group(group_id=53f45b0a49154be9a2c7124b7ee49fa7)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:06.392742 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005abe6c-8b4e-4c86-8a8a-e6e1f1c9e03f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:06.395507 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 191/376] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 22:23:06 2022] GET /identity/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7 => generated 226 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:06.398268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634298ce-7ccc-4c48-a20c-7983b825f61c None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:06.398595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634298ce-7ccc-4c48-a20c-7983b825f61c None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:06.399176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634298ce-7ccc-4c48-a20c-7983b825f61c None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:06.400668 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 192/377] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:06.403683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634298ce-7ccc-4c48-a20c-7983b825f61c None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:06.533951 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8196cb-1939-4ca5-9e25-4382fbea1c97 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:06.588527 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634298ce-7ccc-4c48-a20c-7983b825f61c None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:06.591801 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ecc17-c25c-4111-b453-fe1eae3286e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:06.592757 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ecc17-c25c-4111-b453-fe1eae3286e2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:06.593120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ecc17-c25c-4111-b453-fe1eae3286e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:06.593476 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ecc17-c25c-4111-b453-fe1eae3286e2 None admin] PATH_INFO: `/v3/projects/1aef1dab34da4006bf2529ee1b5120da` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:06.594815 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ecc17-c25c-4111-b453-fe1eae3286e2 None admin] RBAC: Authorizing `identity:get_project(project_id=1aef1dab34da4006bf2529ee1b5120da)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:06.596458 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ecc17-c25c-4111-b453-fe1eae3286e2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:06.598551 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 186/378] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:23:06 2022] GET /identity/v3/projects/1aef1dab34da4006bf2529ee1b5120da => generated 344 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:06.599747 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96700f0e-aaf9-4a41-8b20-87bdf561ca8f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:06.600165 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96700f0e-aaf9-4a41-8b20-87bdf561ca8f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:06.600574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96700f0e-aaf9-4a41-8b20-87bdf561ca8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:06.625910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-96700f0e-aaf9-4a41-8b20-87bdf561ca8f None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:06.627250 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-96700f0e-aaf9-4a41-8b20-87bdf561ca8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:06.637950 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95154a70-ebfd-4f7f-86fe-4b08d47c3d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:06.639341 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95154a70-ebfd-4f7f-86fe-4b08d47c3d3e None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:06.639832 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95154a70-ebfd-4f7f-86fe-4b08d47c3d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:06.640731 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95154a70-ebfd-4f7f-86fe-4b08d47c3d3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:06.642053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95154a70-ebfd-4f7f-86fe-4b08d47c3d3e None admin] RBAC: Adding query filter params (group.id=53f45b0a49154be9a2c7124b7ee49fa7, role.id=61397d4a1ccd4a8ca479f98186ed489b, scope.project.id=6525e5222e3342a3a26a6369518c7577) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:06.642560 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95154a70-ebfd-4f7f-86fe-4b08d47c3d3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:06.644550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:06.646069 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95154a70-ebfd-4f7f-86fe-4b08d47c3d3e None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:06.661423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 193/379] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 22:23:06 2022] GET /identity/v3/role_assignments?group.id=53f45b0a49154be9a2c7124b7ee49fa7&scope.project.id=6525e5222e3342a3a26a6369518c7577&role.id=61397d4a1ccd4a8ca479f98186ed489b => generated 260 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:06.663025 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95154a70-ebfd-4f7f-86fe-4b08d47c3d3e None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:06.775454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 187/380] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:06.822340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:06.825438 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95154a70-ebfd-4f7f-86fe-4b08d47c3d3e None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:06.839749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94677e6d-f18f-433c-b39c-fdeae8b8c8b2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:06.840285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94677e6d-f18f-433c-b39c-fdeae8b8c8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:06.840530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94677e6d-f18f-433c-b39c-fdeae8b8c8b2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:06.841990 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 188/381] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:23:06.850071 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb97dad-11be-40ef-98b4-74083ab31931 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:06.850416 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb97dad-11be-40ef-98b4-74083ab31931 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:06.850681 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb97dad-11be-40ef-98b4-74083ab31931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:06.874087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cbb97dad-11be-40ef-98b4-74083ab31931 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:06.875317 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cbb97dad-11be-40ef-98b4-74083ab31931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:06.889119 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8d9c3-7445-436f-89ee-744b81b54eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:06.889992 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8d9c3-7445-436f-89ee-744b81b54eaa None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:06.890282 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8d9c3-7445-436f-89ee-744b81b54eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:06.890563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8d9c3-7445-436f-89ee-744b81b54eaa None admin] PATH_INFO: `/v3/groups/d888fe884c8443b2961ca5e7fe664e7a` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:06.896139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8d9c3-7445-436f-89ee-744b81b54eaa None admin] RBAC: Authorizing `identity:get_group(group_id=d888fe884c8443b2961ca5e7fe664e7a)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:06.897924 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8d9c3-7445-436f-89ee-744b81b54eaa None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:06.899773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 194/382] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 22:23:06 2022] GET /identity/v3/groups/d888fe884c8443b2961ca5e7fe664e7a => generated 229 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:06.901668 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8d9c3-7445-436f-89ee-744b81b54eaa None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:06.962506 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 189/383] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:23:06.964105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb97dad-11be-40ef-98b4-74083ab31931 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:07.058131 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8d9c3-7445-436f-89ee-744b81b54eaa None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:07.068388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:07.092786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:07.092972 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb97dad-11be-40ef-98b4-74083ab31931 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:07.104021 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84889105-1ddc-4665-9008-a28e5d3f7abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:07.104898 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84889105-1ddc-4665-9008-a28e5d3f7abf None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:07.105190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84889105-1ddc-4665-9008-a28e5d3f7abf None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:07.105473 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84889105-1ddc-4665-9008-a28e5d3f7abf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:07.118366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84889105-1ddc-4665-9008-a28e5d3f7abf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:07.126351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-84889105-1ddc-4665-9008-a28e5d3f7abf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:23:07.126351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:07.126351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:07.126351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:07.126351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:07.126351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:07.126351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:07.126351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:07.126351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:07.126351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:07.126351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:07.126351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:07.126351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:07.126351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:07.126351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:07.126351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:07.127294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:07.127294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:07.127294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:07.127294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:07.127294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:07.127294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:07.127294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:07.127294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:07.127294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:07.127294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:07.127294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:07.127294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:07.127294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:07.127294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:07.127294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:07.127294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:07.128091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:07.128091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:07.128091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:07.128091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:07.128091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:07.128091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:07.128091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:07.128091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:07.128091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:07.128091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:07.128091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:23:07.128091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:07.128091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 195/384] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:07.128732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84889105-1ddc-4665-9008-a28e5d3f7abf None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:07.148441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ed8a8-d580-4f1f-9d9e-a3585b439ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:07.149798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ed8a8-d580-4f1f-9d9e-a3585b439ced None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:07.150233 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ed8a8-d580-4f1f-9d9e-a3585b439ced None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:07.150826 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ed8a8-d580-4f1f-9d9e-a3585b439ced None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:07.151960 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ed8a8-d580-4f1f-9d9e-a3585b439ced None admin] RBAC: Adding query filter params (group.id=d888fe884c8443b2961ca5e7fe664e7a, role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.project.id=1aef1dab34da4006bf2529ee1b5120da) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:07.152578 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ed8a8-d580-4f1f-9d9e-a3585b439ced None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:07.154518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ed8a8-d580-4f1f-9d9e-a3585b439ced None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:07.161761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 190/385] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 22:23:06 2022] GET /identity/v3/role_assignments?group.id=d888fe884c8443b2961ca5e7fe664e7a&scope.project.id=1aef1dab34da4006bf2529ee1b5120da&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:07.163139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ed8a8-d580-4f1f-9d9e-a3585b439ced None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:07.257004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84889105-1ddc-4665-9008-a28e5d3f7abf None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:07.316289 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425814eb-9507-42d2-8bf4-7a38ba362823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:07.317277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425814eb-9507-42d2-8bf4-7a38ba362823 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:07.317669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425814eb-9507-42d2-8bf4-7a38ba362823 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:07.318060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425814eb-9507-42d2-8bf4-7a38ba362823 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:07.330258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ed8a8-d580-4f1f-9d9e-a3585b439ced None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:07.333640 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425814eb-9507-42d2-8bf4-7a38ba362823 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:07.343953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-425814eb-9507-42d2-8bf4-7a38ba362823 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:23:07.343953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:07.343953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:07.343953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:07.343953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:07.343953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:07.343953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:07.343953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:07.343953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:07.343953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:07.343953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:07.343953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:07.343953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:07.343953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:07.343953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:07.343953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:07.344900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:07.344900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:07.344900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:07.344900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:07.344900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:07.344900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:07.344900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:07.344900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:07.344900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:07.344900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:07.344900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:07.344900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:07.344900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:07.344900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:07.344900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:07.344900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:07.345703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:07.345703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:07.345703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:07.345703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:07.345703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:07.345703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:07.345703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:07.345703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:07.345703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:07.345703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:07.345703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:23:07.345703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:07.346344 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 196/386] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:23:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:07.354916 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425814eb-9507-42d2-8bf4-7a38ba362823 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:07.415968 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:07.430317 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb61ba-b353-45aa-828d-284b5d86f504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:07.432183 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb61ba-b353-45aa-828d-284b5d86f504 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:07.432686 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb61ba-b353-45aa-828d-284b5d86f504 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:07.433466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb61ba-b353-45aa-828d-284b5d86f504 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:07.434757 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb61ba-b353-45aa-828d-284b5d86f504 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:07.435734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb61ba-b353-45aa-828d-284b5d86f504 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:07.437867 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb61ba-b353-45aa-828d-284b5d86f504 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:07.450679 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 191/387] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:23:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:07.457605 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb61ba-b353-45aa-828d-284b5d86f504 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:07.470229 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:07.493581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425814eb-9507-42d2-8bf4-7a38ba362823 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:07.532933 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92571f69-34f8-4fa4-b515-ddb634689210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:07.533601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92571f69-34f8-4fa4-b515-ddb634689210 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:07.533948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92571f69-34f8-4fa4-b515-ddb634689210 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:07.534233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92571f69-34f8-4fa4-b515-ddb634689210 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:07.535593 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92571f69-34f8-4fa4-b515-ddb634689210 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:07.535916 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92571f69-34f8-4fa4-b515-ddb634689210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:07.537889 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92571f69-34f8-4fa4-b515-ddb634689210 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:07.551016 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 197/388] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:07.554557 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92571f69-34f8-4fa4-b515-ddb634689210 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:07.598477 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb61ba-b353-45aa-828d-284b5d86f504 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:07.634254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7387fcc-bfe8-4144-a80e-b7febc51666e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:07.635308 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7387fcc-bfe8-4144-a80e-b7febc51666e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:07.635667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7387fcc-bfe8-4144-a80e-b7febc51666e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:07.636026 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7387fcc-bfe8-4144-a80e-b7febc51666e None admin] PATH_INFO: `/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:07.637403 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7387fcc-bfe8-4144-a80e-b7febc51666e None admin] RBAC: Authorizing `identity:get_group(group_id=53f45b0a49154be9a2c7124b7ee49fa7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:07.639064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7387fcc-bfe8-4144-a80e-b7febc51666e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:07.641292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 192/389] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 22:23:07 2022] GET /identity/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:07.651238 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7387fcc-bfe8-4144-a80e-b7febc51666e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:07.710063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92571f69-34f8-4fa4-b515-ddb634689210 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:07.720975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:07.754954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee90da-66d9-44b6-ac51-fa36f3be28ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:07.757501 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee90da-66d9-44b6-ac51-fa36f3be28ed None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:07.758868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee90da-66d9-44b6-ac51-fa36f3be28ed None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:07.759967 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee90da-66d9-44b6-ac51-fa36f3be28ed None admin] PATH_INFO: `/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:07.763979 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee90da-66d9-44b6-ac51-fa36f3be28ed None admin] RBAC: Authorizing `identity:get_group(group_id=53f45b0a49154be9a2c7124b7ee49fa7)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:07.765595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee90da-66d9-44b6-ac51-fa36f3be28ed None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:07.768804 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 198/390] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 22:23:07 2022] GET /identity/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:07.775380 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee90da-66d9-44b6-ac51-fa36f3be28ed None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:07.783303 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:07.852022 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7387fcc-bfe8-4144-a80e-b7febc51666e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:07.916786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155f4ff-6153-4136-8409-c45ab0af7bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:07.917661 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155f4ff-6153-4136-8409-c45ab0af7bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:07.917954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155f4ff-6153-4136-8409-c45ab0af7bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:07.918240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155f4ff-6153-4136-8409-c45ab0af7bc0 None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:07.920295 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155f4ff-6153-4136-8409-c45ab0af7bc0 None admin] RBAC: Authorizing `identity:get_project(project_id=53390afb4a404df79ab2d9ae31e8d058)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:07.921736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155f4ff-6153-4136-8409-c45ab0af7bc0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:07.923480 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 193/391] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:23:07 2022] GET /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058 => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:07.929516 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee90da-66d9-44b6-ac51-fa36f3be28ed None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:07.931766 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155f4ff-6153-4136-8409-c45ab0af7bc0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:07.996796 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1424d-b86c-4c52-9c0c-1b392bf0d286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:07.997729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1424d-b86c-4c52-9c0c-1b392bf0d286 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:07.998031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1424d-b86c-4c52-9c0c-1b392bf0d286 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:07.998480 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1424d-b86c-4c52-9c0c-1b392bf0d286 None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:07.999728 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1424d-b86c-4c52-9c0c-1b392bf0d286 None admin] RBAC: Authorizing `identity:get_project(project_id=53390afb4a404df79ab2d9ae31e8d058)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:08.001173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1424d-b86c-4c52-9c0c-1b392bf0d286 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:08.007957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 199/392] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:23:07 2022] GET /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:08.009899 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1424d-b86c-4c52-9c0c-1b392bf0d286 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:08.077974 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:08.148911 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155f4ff-6153-4136-8409-c45ab0af7bc0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:08.185276 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1424d-b86c-4c52-9c0c-1b392bf0d286 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:08.207990 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a9a00-6691-43e3-98c2-5ce8fc8f5498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:08.208878 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a9a00-6691-43e3-98c2-5ce8fc8f5498 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:08.209164 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a9a00-6691-43e3-98c2-5ce8fc8f5498 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:08.209446 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a9a00-6691-43e3-98c2-5ce8fc8f5498 None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058/groups/53f45b0a49154be9a2c7124b7ee49fa7/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:08.216223 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a9a00-6691-43e3-98c2-5ce8fc8f5498 None admin] RBAC: Authorizing `identity:create_grant(project_id=53390afb4a404df79ab2d9ae31e8d058, group_id=53f45b0a49154be9a2c7124b7ee49fa7, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:08.217658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a9a00-6691-43e3-98c2-5ce8fc8f5498 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:08.242248 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 194/393] 10.222.0.43 () {58 vars in 1442 bytes} [Thu Jun 23 22:23:07 2022] PUT /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058/groups/53f45b0a49154be9a2c7124b7ee49fa7/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:23:08.250540 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533db100-fb0e-4c5f-82eb-30c7914258f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:08.251578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533db100-fb0e-4c5f-82eb-30c7914258f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:08.251865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533db100-fb0e-4c5f-82eb-30c7914258f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:08.252148 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533db100-fb0e-4c5f-82eb-30c7914258f9 None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058/groups/53f45b0a49154be9a2c7124b7ee49fa7/roles/61397d4a1ccd4a8ca479f98186ed489b` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:08.255522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533db100-fb0e-4c5f-82eb-30c7914258f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=53390afb4a404df79ab2d9ae31e8d058, group_id=53f45b0a49154be9a2c7124b7ee49fa7, role_id=61397d4a1ccd4a8ca479f98186ed489b)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:08.257545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533db100-fb0e-4c5f-82eb-30c7914258f9 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:08.276564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 200/394] 10.222.0.43 () {58 vars in 1442 bytes} [Thu Jun 23 22:23:08 2022] PUT /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058/groups/53f45b0a49154be9a2c7124b7ee49fa7/roles/61397d4a1ccd4a8ca479f98186ed489b => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:23:08.886900 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67455ab-0848-454a-bafc-7011ba7bfcec None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:08.887418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67455ab-0848-454a-bafc-7011ba7bfcec None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:08.887519 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67455ab-0848-454a-bafc-7011ba7bfcec None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:08.889184 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 195/395] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:08.894819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3cb5e-db87-4edf-9c8b-9afae619c5c7 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:08.895370 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3cb5e-db87-4edf-9c8b-9afae619c5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:08.895871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3cb5e-db87-4edf-9c8b-9afae619c5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:08.923943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-a3b3cb5e-db87-4edf-9c8b-9afae619c5c7 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:08.925513 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-a3b3cb5e-db87-4edf-9c8b-9afae619c5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:09.013452 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 201/396] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:23:09.052884 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67455ab-0848-454a-bafc-7011ba7bfcec None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:09.056997 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:09.164664 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64a1fc-b1b3-4d30-a6d0-9b575ec875e9 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:09.165142 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64a1fc-b1b3-4d30-a6d0-9b575ec875e9 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:09.165142 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64a1fc-b1b3-4d30-a6d0-9b575ec875e9 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:09.166791 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 202/397] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:09.171682 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cb975-8303-4d7a-952d-8dd53aea474a None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:09.171972 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cb975-8303-4d7a-952d-8dd53aea474a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:09.172252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cb975-8303-4d7a-952d-8dd53aea474a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:09.203385 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-799cb975-8303-4d7a-952d-8dd53aea474a None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:09.204692 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-799cb975-8303-4d7a-952d-8dd53aea474a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:09.262361 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67455ab-0848-454a-bafc-7011ba7bfcec None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:09.264865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:09.314906 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8329c95-92d5-4f19-9a78-f249cccf86e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:09.315736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8329c95-92d5-4f19-9a78-f249cccf86e1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:09.316021 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8329c95-92d5-4f19-9a78-f249cccf86e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:09.316303 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8329c95-92d5-4f19-9a78-f249cccf86e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:09.335094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8329c95-92d5-4f19-9a78-f249cccf86e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:09.348153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a8329c95-92d5-4f19-9a78-f249cccf86e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:23:09.348153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:09.348153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:09.348153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:09.348153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:09.348153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:09.348153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:09.348153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:09.348153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:09.348153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:09.348153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:09.348153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:09.348153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:09.348153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:09.348153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:09.348153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:09.349075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:09.349075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:09.349075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:09.349075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:09.349075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:09.349075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:09.349075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:09.349075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:09.349075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:09.349075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:09.349075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:09.349075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:09.349075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:09.349075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:09.349075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:09.349075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:09.349862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:09.349862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:09.349862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:09.349862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:09.349862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:09.349862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:09.349862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:09.349862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:09.349862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:09.349862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:09.349862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:23:09.349862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:09.355899 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 196/398] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:09.362799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8329c95-92d5-4f19-9a78-f249cccf86e1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:09.367727 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 203/399] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:09.404735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cb975-8303-4d7a-952d-8dd53aea474a None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:09.477592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:09.553099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8329c95-92d5-4f19-9a78-f249cccf86e1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:09.556880 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cb975-8303-4d7a-952d-8dd53aea474a None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:09.611477 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4511ba-01b4-47e4-9c9d-1930da9937b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:09.612146 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4511ba-01b4-47e4-9c9d-1930da9937b5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:09.612432 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4511ba-01b4-47e4-9c9d-1930da9937b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:09.612717 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4511ba-01b4-47e4-9c9d-1930da9937b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:09.613463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4511ba-01b4-47e4-9c9d-1930da9937b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:09.613772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4511ba-01b4-47e4-9c9d-1930da9937b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:09.615187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4511ba-01b4-47e4-9c9d-1930da9937b5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:09.618581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42050c6a-bd97-4a49-beb0-fb234c342fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:09.619428 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42050c6a-bd97-4a49-beb0-fb234c342fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:09.619711 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42050c6a-bd97-4a49-beb0-fb234c342fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:09.619988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42050c6a-bd97-4a49-beb0-fb234c342fd6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:09.637723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 197/400] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:23:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:09.643101 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4511ba-01b4-47e4-9c9d-1930da9937b5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:09.643985 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42050c6a-bd97-4a49-beb0-fb234c342fd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:09.655398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-42050c6a-bd97-4a49-beb0-fb234c342fd6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:23:09.655398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:09.655398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:09.655398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:09.655398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:09.655398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:09.655398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:09.655398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:09.655398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:09.655398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:09.655398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:09.655398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:09.655398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:09.655398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:09.655398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:09.655398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:09.656308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:09.656308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:09.656308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:09.656308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:09.656308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:09.656308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:09.656308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:09.656308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:09.656308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:09.656308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:09.656308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:09.656308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:09.656308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:09.656308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:09.656308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:09.656308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:09.657137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:09.657137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:09.657137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:09.657137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:09.657137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:09.657137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:09.657137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:09.657137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:09.657137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:09.657137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:09.657137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:23:09.657137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:09.657137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 204/401] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:23:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:09.661547 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42050c6a-bd97-4a49-beb0-fb234c342fd6 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:09.691709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:09.793106 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42050c6a-bd97-4a49-beb0-fb234c342fd6 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:09.863582 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ba773-4435-40ba-83cb-aa031a953332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:09.865927 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ba773-4435-40ba-83cb-aa031a953332 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:09.868052 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ba773-4435-40ba-83cb-aa031a953332 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:09.868536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ba773-4435-40ba-83cb-aa031a953332 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:09.869510 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ba773-4435-40ba-83cb-aa031a953332 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:09.871237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ba773-4435-40ba-83cb-aa031a953332 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:09.872843 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ba773-4435-40ba-83cb-aa031a953332 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:09.874033 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4511ba-01b4-47e4-9c9d-1930da9937b5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:09.884413 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 205/402] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:09.887247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5237cca-5104-4b36-9e4b-2b3c40e121a8 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:09.887762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5237cca-5104-4b36-9e4b-2b3c40e121a8 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:09.888234 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5237cca-5104-4b36-9e4b-2b3c40e121a8 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:09.890560 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 206/403] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:09.891599 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5237cca-5104-4b36-9e4b-2b3c40e121a8 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:09.910542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:09.926333 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7f2b0-c82c-4d93-8c00-3034b6e78d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:09.927650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7f2b0-c82c-4d93-8c00-3034b6e78d6b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:09.929072 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7f2b0-c82c-4d93-8c00-3034b6e78d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:09.929789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7f2b0-c82c-4d93-8c00-3034b6e78d6b None admin] PATH_INFO: `/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:09.931732 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7f2b0-c82c-4d93-8c00-3034b6e78d6b None admin] RBAC: Authorizing `identity:get_group(group_id=53f45b0a49154be9a2c7124b7ee49fa7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:09.933416 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7f2b0-c82c-4d93-8c00-3034b6e78d6b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:09.935745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 198/404] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 22:23:09 2022] GET /identity/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7 => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:09.937175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f73d5-b144-408f-8422-93bad2b1d941 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:09.937655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f73d5-b144-408f-8422-93bad2b1d941 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:09.938179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f73d5-b144-408f-8422-93bad2b1d941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:09.962670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-129f73d5-b144-408f-8422-93bad2b1d941 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:09.964712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-129f73d5-b144-408f-8422-93bad2b1d941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:10.003243 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:10.029339 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5237cca-5104-4b36-9e4b-2b3c40e121a8 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:10.043496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 199/405] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:10.044349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f73d5-b144-408f-8422-93bad2b1d941 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:10.077384 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdad8da-c29f-4c9b-b0a2-ffc5b310fa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:10.079255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdad8da-c29f-4c9b-b0a2-ffc5b310fa6d None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:10.079740 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdad8da-c29f-4c9b-b0a2-ffc5b310fa6d None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:10.080456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdad8da-c29f-4c9b-b0a2-ffc5b310fa6d None admin] PATH_INFO: `/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:10.082500 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdad8da-c29f-4c9b-b0a2-ffc5b310fa6d None admin] RBAC: Authorizing `identity:get_group(group_id=53f45b0a49154be9a2c7124b7ee49fa7)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:10.084422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdad8da-c29f-4c9b-b0a2-ffc5b310fa6d None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:10.087638 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 207/406] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 22:23:09 2022] GET /identity/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:10.088712 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdad8da-c29f-4c9b-b0a2-ffc5b310fa6d None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:10.175411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f73d5-b144-408f-8422-93bad2b1d941 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:10.206576 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:10.208191 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdad8da-c29f-4c9b-b0a2-ffc5b310fa6d None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:10.223566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5bac2-9fe7-4e80-bb5f-1ac65c6b152b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:10.225390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5bac2-9fe7-4e80-bb5f-1ac65c6b152b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:10.225862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5bac2-9fe7-4e80-bb5f-1ac65c6b152b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:10.226322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5bac2-9fe7-4e80-bb5f-1ac65c6b152b None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:10.228263 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5bac2-9fe7-4e80-bb5f-1ac65c6b152b None admin] RBAC: Authorizing `identity:get_project(project_id=6525e5222e3342a3a26a6369518c7577)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:10.229864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5bac2-9fe7-4e80-bb5f-1ac65c6b152b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:10.237525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 200/407] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:23:10 2022] GET /identity/v3/projects/6525e5222e3342a3a26a6369518c7577 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:10.237525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5bac2-9fe7-4e80-bb5f-1ac65c6b152b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:10.251724 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f312514-4fb8-4de6-91d4-9fd328e46909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:10.252775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f312514-4fb8-4de6-91d4-9fd328e46909 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:10.253254 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f312514-4fb8-4de6-91d4-9fd328e46909 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:10.253773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f312514-4fb8-4de6-91d4-9fd328e46909 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:10.265601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f312514-4fb8-4de6-91d4-9fd328e46909 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:10.274604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-4f312514-4fb8-4de6-91d4-9fd328e46909 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:23:10.274604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:10.274604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:10.274604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:10.274604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:10.274604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:10.274604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:10.274604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:10.274604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:10.274604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:10.274604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:10.274604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:10.274604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:10.274604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:10.274604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:10.274604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:10.275472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:10.275472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:10.275472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:10.275472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:10.275472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:10.275472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:10.275472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:10.275472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:10.275472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:10.275472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:10.275472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:10.275472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:10.275472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:10.275472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:10.275472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:10.275472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:10.276265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:10.276265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:10.276265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:10.276265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:10.276265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:10.276265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:10.276265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:10.276265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:10.276265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:10.276265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:10.276265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:23:10.276265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:10.278902 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 208/408] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:23:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:10.279918 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f312514-4fb8-4de6-91d4-9fd328e46909 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:10.296968 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:10.342793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5bac2-9fe7-4e80-bb5f-1ac65c6b152b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:10.379268 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba6165-87af-4ccb-885b-b2214124a680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:10.380184 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba6165-87af-4ccb-885b-b2214124a680 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:10.380475 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba6165-87af-4ccb-885b-b2214124a680 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:10.380760 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba6165-87af-4ccb-885b-b2214124a680 None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:10.382554 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba6165-87af-4ccb-885b-b2214124a680 None admin] RBAC: Authorizing `identity:get_project(project_id=6525e5222e3342a3a26a6369518c7577)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:10.384104 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba6165-87af-4ccb-885b-b2214124a680 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:10.385833 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 201/409] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:23:10 2022] GET /identity/v3/projects/6525e5222e3342a3a26a6369518c7577 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:10.386656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba6165-87af-4ccb-885b-b2214124a680 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:10.393287 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f312514-4fb8-4de6-91d4-9fd328e46909 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:10.431546 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2aea7-b3b0-4055-ad55-7531dfca350d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:10.432889 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2aea7-b3b0-4055-ad55-7531dfca350d None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:10.433368 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2aea7-b3b0-4055-ad55-7531dfca350d None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:10.433857 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2aea7-b3b0-4055-ad55-7531dfca350d None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577/groups/53f45b0a49154be9a2c7124b7ee49fa7/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:10.436296 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2aea7-b3b0-4055-ad55-7531dfca350d None admin] RBAC: Authorizing `identity:create_grant(project_id=6525e5222e3342a3a26a6369518c7577, group_id=53f45b0a49154be9a2c7124b7ee49fa7, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:10.437935 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2aea7-b3b0-4055-ad55-7531dfca350d None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:10.449723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:10.462951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 209/410] 10.222.0.43 () {58 vars in 1442 bytes} [Thu Jun 23 22:23:10 2022] PUT /identity/v3/projects/6525e5222e3342a3a26a6369518c7577/groups/53f45b0a49154be9a2c7124b7ee49fa7/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:23:10.464568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2aea7-b3b0-4055-ad55-7531dfca350d None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:10.500182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba6165-87af-4ccb-885b-b2214124a680 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:10.538566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05539a1b-e937-4ae2-9b27-f7aba74ee168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:10.539404 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05539a1b-e937-4ae2-9b27-f7aba74ee168 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:10.539696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05539a1b-e937-4ae2-9b27-f7aba74ee168 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:10.540021 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05539a1b-e937-4ae2-9b27-f7aba74ee168 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:10.540783 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05539a1b-e937-4ae2-9b27-f7aba74ee168 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:10.541149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05539a1b-e937-4ae2-9b27-f7aba74ee168 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:10.542547 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05539a1b-e937-4ae2-9b27-f7aba74ee168 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:10.552205 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 202/411] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:10.557390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05539a1b-e937-4ae2-9b27-f7aba74ee168 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:10.614197 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:10.683306 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05539a1b-e937-4ae2-9b27-f7aba74ee168 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:10.717013 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2aea7-b3b0-4055-ad55-7531dfca350d None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:10.721702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:10.725731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6ce49-726a-4883-b5bf-5b0b8bc5249d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:10.726797 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6ce49-726a-4883-b5bf-5b0b8bc5249d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:10.727114 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6ce49-726a-4883-b5bf-5b0b8bc5249d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:10.727393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6ce49-726a-4883-b5bf-5b0b8bc5249d None admin] PATH_INFO: `/v3/groups/d888fe884c8443b2961ca5e7fe664e7a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:10.728600 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6ce49-726a-4883-b5bf-5b0b8bc5249d None admin] RBAC: Authorizing `identity:get_group(group_id=d888fe884c8443b2961ca5e7fe664e7a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:10.730037 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6ce49-726a-4883-b5bf-5b0b8bc5249d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:10.732063 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 203/412] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 22:23:10 2022] GET /identity/v3/groups/d888fe884c8443b2961ca5e7fe664e7a => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:10.736963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6ce49-726a-4883-b5bf-5b0b8bc5249d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:10.769893 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ce201-c2cd-4c53-8d22-7dcb3e0554d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:10.770792 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ce201-c2cd-4c53-8d22-7dcb3e0554d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:10.771142 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ce201-c2cd-4c53-8d22-7dcb3e0554d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:10.771426 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ce201-c2cd-4c53-8d22-7dcb3e0554d0 None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577/groups/53f45b0a49154be9a2c7124b7ee49fa7/roles/61397d4a1ccd4a8ca479f98186ed489b` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:10.774377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ce201-c2cd-4c53-8d22-7dcb3e0554d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6525e5222e3342a3a26a6369518c7577, group_id=53f45b0a49154be9a2c7124b7ee49fa7, role_id=61397d4a1ccd4a8ca479f98186ed489b)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:10.775876 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ce201-c2cd-4c53-8d22-7dcb3e0554d0 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:10.794344 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 210/413] 10.222.0.43 () {58 vars in 1442 bytes} [Thu Jun 23 22:23:10 2022] PUT /identity/v3/projects/6525e5222e3342a3a26a6369518c7577/groups/53f45b0a49154be9a2c7124b7ee49fa7/roles/61397d4a1ccd4a8ca479f98186ed489b => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:23:10.868335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6ce49-726a-4883-b5bf-5b0b8bc5249d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:10.908555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79f9d3-fa4f-4428-8b81-848179bf8459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:10.909906 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79f9d3-fa4f-4428-8b81-848179bf8459 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:10.910393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79f9d3-fa4f-4428-8b81-848179bf8459 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:10.910910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79f9d3-fa4f-4428-8b81-848179bf8459 None admin] PATH_INFO: `/v3/projects/1aef1dab34da4006bf2529ee1b5120da` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:10.913374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79f9d3-fa4f-4428-8b81-848179bf8459 None admin] RBAC: Authorizing `identity:get_project(project_id=1aef1dab34da4006bf2529ee1b5120da)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:10.915099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79f9d3-fa4f-4428-8b81-848179bf8459 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:10.917438 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 204/414] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:23:10 2022] GET /identity/v3/projects/1aef1dab34da4006bf2529ee1b5120da => generated 344 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:10.924534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ce201-c2cd-4c53-8d22-7dcb3e0554d0 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:11.012500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980436e0-eead-4b02-bc39-b35c9db9cc4a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:11.013109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980436e0-eead-4b02-bc39-b35c9db9cc4a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:11.013397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980436e0-eead-4b02-bc39-b35c9db9cc4a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:11.014989 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 205/415] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:11.020454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc928c6b-4982-46ae-b42b-d7c8881e4945 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:11.020747 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc928c6b-4982-46ae-b42b-d7c8881e4945 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:11.021029 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc928c6b-4982-46ae-b42b-d7c8881e4945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:11.042477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ce201-c2cd-4c53-8d22-7dcb3e0554d0 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:11.042911 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-dc928c6b-4982-46ae-b42b-d7c8881e4945 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:11.044139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-dc928c6b-4982-46ae-b42b-d7c8881e4945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:11.079395 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d345d3-3b44-4b16-9fbf-6433996241e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:11.080293 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d345d3-3b44-4b16-9fbf-6433996241e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:11.080584 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d345d3-3b44-4b16-9fbf-6433996241e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:11.080866 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d345d3-3b44-4b16-9fbf-6433996241e9 None admin] PATH_INFO: `/v3/projects/1aef1dab34da4006bf2529ee1b5120da/groups/d888fe884c8443b2961ca5e7fe664e7a/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:11.082859 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d345d3-3b44-4b16-9fbf-6433996241e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=1aef1dab34da4006bf2529ee1b5120da, group_id=d888fe884c8443b2961ca5e7fe664e7a, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:11.084766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d345d3-3b44-4b16-9fbf-6433996241e9 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:11.088027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:11.101131 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 211/416] 10.222.0.43 () {58 vars in 1442 bytes} [Thu Jun 23 22:23:10 2022] PUT /identity/v3/projects/1aef1dab34da4006bf2529ee1b5120da/groups/d888fe884c8443b2961ca5e7fe664e7a/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:23:11.103022 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203f436-a9ec-4683-8be3-125134746923 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:11.103332 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203f436-a9ec-4683-8be3-125134746923 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:11.103734 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203f436-a9ec-4683-8be3-125134746923 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:11.104943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 212/417] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:11.110584 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9787f-8355-48aa-a86e-c0a9e0ca84ac None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:11.110875 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9787f-8355-48aa-a86e-c0a9e0ca84ac None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:11.111250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9787f-8355-48aa-a86e-c0a9e0ca84ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:11.120661 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:11.134003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-35e9787f-8355-48aa-a86e-c0a9e0ca84ac None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:11.135284 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-35e9787f-8355-48aa-a86e-c0a9e0ca84ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:11.173347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 206/418] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:11.212505 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 213/419] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:23:11.213017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc928c6b-4982-46ae-b42b-d7c8881e4945 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:11.236104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9787f-8355-48aa-a86e-c0a9e0ca84ac None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:11.343862 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9787f-8355-48aa-a86e-c0a9e0ca84ac None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:11.370150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:11.382501 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6762a5c-945e-40d2-9fad-3d5b4e27651d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:11.382987 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6762a5c-945e-40d2-9fad-3d5b4e27651d None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:11.383276 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6762a5c-945e-40d2-9fad-3d5b4e27651d None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:11.383555 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6762a5c-945e-40d2-9fad-3d5b4e27651d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:11.386364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc928c6b-4982-46ae-b42b-d7c8881e4945 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:11.394451 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6762a5c-945e-40d2-9fad-3d5b4e27651d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:11.403702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-f6762a5c-945e-40d2-9fad-3d5b4e27651d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:23:11.403702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:11.403702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:11.403702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:11.403702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:11.403702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:11.403702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:11.403702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:11.403702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:11.403702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:11.403702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:11.403702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:11.403702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:11.403702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:11.403702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:11.403702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:11.404645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:11.404645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:11.404645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:11.404645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:11.404645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:11.404645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:11.404645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:11.404645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:11.404645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:11.404645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:11.404645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:11.404645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:11.404645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:11.404645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:11.404645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:11.404645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:11.405425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:11.405425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:11.405425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:11.405425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:11.405425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:11.405425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:11.405425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:11.405425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:11.405425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:11.405425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:11.405425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:23:11.405425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:11.405425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 214/420] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:11.410913 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6762a5c-945e-40d2-9fad-3d5b4e27651d None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:11.431208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ac6a2-06a9-42fb-ba18-fb7ba1f1ce1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:11.432469 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ac6a2-06a9-42fb-ba18-fb7ba1f1ce1d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:11.432953 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ac6a2-06a9-42fb-ba18-fb7ba1f1ce1d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:11.433451 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ac6a2-06a9-42fb-ba18-fb7ba1f1ce1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:11.446182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ac6a2-06a9-42fb-ba18-fb7ba1f1ce1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:11.455044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f28ac6a2-06a9-42fb-ba18-fb7ba1f1ce1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:23:11.455044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:11.455044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:11.455044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:11.455044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:11.455044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:11.455044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:11.455044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:11.455044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:11.455044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:11.455044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:11.455044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:11.455044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:11.455044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:11.455044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:11.455044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:11.455955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:11.455955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:11.455955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:11.455955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:11.455955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:11.455955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:11.455955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:11.455955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:11.455955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:11.455955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:11.455955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:11.455955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:11.455955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:11.455955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:11.455955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:11.455955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:11.456788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:11.456788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:11.456788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:11.456788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:11.456788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:11.456788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:11.456788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:11.456788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:11.456788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:11.456788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:11.456788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:23:11.456788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:11.459326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 207/421] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:23:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:11.466537 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ac6a2-06a9-42fb-ba18-fb7ba1f1ce1d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:11.517054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6762a5c-945e-40d2-9fad-3d5b4e27651d None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:11.520693 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:11.551782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bc077-57a5-44de-a91e-27b6a8c90923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:11.552461 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bc077-57a5-44de-a91e-27b6a8c90923 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:11.552744 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bc077-57a5-44de-a91e-27b6a8c90923 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:11.553020 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bc077-57a5-44de-a91e-27b6a8c90923 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:11.554470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bc077-57a5-44de-a91e-27b6a8c90923 None admin] RBAC: Adding query filter params (name=member) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:11.554703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bc077-57a5-44de-a91e-27b6a8c90923 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:11.556113 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bc077-57a5-44de-a91e-27b6a8c90923 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:11.565384 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 215/422] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:23:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:11.571484 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bc077-57a5-44de-a91e-27b6a8c90923 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:11.587027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ac6a2-06a9-42fb-ba18-fb7ba1f1ce1d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:11.619741 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:11.628163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03d345-e593-4516-94d3-b6f2b69d6f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:11.629217 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03d345-e593-4516-94d3-b6f2b69d6f61 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:11.629821 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03d345-e593-4516-94d3-b6f2b69d6f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:11.630298 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03d345-e593-4516-94d3-b6f2b69d6f61 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:11.632086 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03d345-e593-4516-94d3-b6f2b69d6f61 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:11.632934 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03d345-e593-4516-94d3-b6f2b69d6f61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:11.634648 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03d345-e593-4516-94d3-b6f2b69d6f61 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:11.644165 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 208/423] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:11.650365 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03d345-e593-4516-94d3-b6f2b69d6f61 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:11.677732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bc077-57a5-44de-a91e-27b6a8c90923 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:11.715103 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fa65a-f0a1-4b82-9488-6e17b0787e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:11.715812 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fa65a-f0a1-4b82-9488-6e17b0787e36 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:11.716099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fa65a-f0a1-4b82-9488-6e17b0787e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:11.716374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fa65a-f0a1-4b82-9488-6e17b0787e36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:11.767310 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03d345-e593-4516-94d3-b6f2b69d6f61 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:11.778512 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:11.786773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fa65a-f0a1-4b82-9488-6e17b0787e36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:11.788876 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fa65a-f0a1-4b82-9488-6e17b0787e36 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:11.810507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7dc2f-7d85-4f5f-8d8e-0e9d806449bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:11.811628 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7dc2f-7d85-4f5f-8d8e-0e9d806449bb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:11.812134 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7dc2f-7d85-4f5f-8d8e-0e9d806449bb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:11.812612 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7dc2f-7d85-4f5f-8d8e-0e9d806449bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:11.844571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:11.897474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 216/424] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:11.903194 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fa65a-f0a1-4b82-9488-6e17b0787e36 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:11.993960 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7dc2f-7d85-4f5f-8d8e-0e9d806449bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:11.993960 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae7dc2f-7d85-4f5f-8d8e-0e9d806449bb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:12.005473 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fa65a-f0a1-4b82-9488-6e17b0787e36 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:12.072196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1ff1d-06c2-4d2f-bd59-c079d5c5132f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:12.073082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1ff1d-06c2-4d2f-bd59-c079d5c5132f None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:12.073372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1ff1d-06c2-4d2f-bd59-c079d5c5132f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:12.073660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1ff1d-06c2-4d2f-bd59-c079d5c5132f None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:12.075668 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1ff1d-06c2-4d2f-bd59-c079d5c5132f None admin] RBAC: Authorizing `identity:get_project(project_id=53390afb4a404df79ab2d9ae31e8d058)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:12.078423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1ff1d-06c2-4d2f-bd59-c079d5c5132f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:12.083697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 217/425] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:23:11 2022] GET /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:12.096603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1ff1d-06c2-4d2f-bd59-c079d5c5132f None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:12.118239 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:12.181870 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 209/426] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 22:23:12.187382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7dc2f-7d85-4f5f-8d8e-0e9d806449bb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:12.199428 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:12.271168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1ff1d-06c2-4d2f-bd59-c079d5c5132f None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:12.321782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a78ee9-a296-4a2d-ae80-6e81646f60d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:12.322649 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a78ee9-a296-4a2d-ae80-6e81646f60d2 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:12.322951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a78ee9-a296-4a2d-ae80-6e81646f60d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:12.323227 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a78ee9-a296-4a2d-ae80-6e81646f60d2 None admin] PATH_INFO: `/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:12.324465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a78ee9-a296-4a2d-ae80-6e81646f60d2 None admin] RBAC: Authorizing `identity:get_group(group_id=53f45b0a49154be9a2c7124b7ee49fa7)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:12.325921 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a78ee9-a296-4a2d-ae80-6e81646f60d2 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:12.328008 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 218/427] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 22:23:12 2022] GET /identity/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:12.333703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a78ee9-a296-4a2d-ae80-6e81646f60d2 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:12.409088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae7dc2f-7d85-4f5f-8d8e-0e9d806449bb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:12.419321 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:12.438021 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a78ee9-a296-4a2d-ae80-6e81646f60d2 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:12.476547 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9871a0-4013-499a-a794-4fc449ebb07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:12.477225 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9871a0-4013-499a-a794-4fc449ebb07f None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:12.477513 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9871a0-4013-499a-a794-4fc449ebb07f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:12.477792 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9871a0-4013-499a-a794-4fc449ebb07f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:12.478626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9871a0-4013-499a-a794-4fc449ebb07f None admin] RBAC: Adding query filter params (group.id=53f45b0a49154be9a2c7124b7ee49fa7, role.id=1baba2239751440eb25af0d0fd2d072d, scope.project.id=53390afb4a404df79ab2d9ae31e8d058) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:12.478947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9871a0-4013-499a-a794-4fc449ebb07f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:12.480615 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9871a0-4013-499a-a794-4fc449ebb07f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:12.487143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 219/428] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 22:23:12 2022] GET /identity/v3/role_assignments?group.id=53f45b0a49154be9a2c7124b7ee49fa7&scope.project.id=53390afb4a404df79ab2d9ae31e8d058&role.id=1baba2239751440eb25af0d0fd2d072d => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:12.487790 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180271eb-0d9e-4a6d-a940-c240a72a7edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:12.488684 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180271eb-0d9e-4a6d-a940-c240a72a7edc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:12.488976 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180271eb-0d9e-4a6d-a940-c240a72a7edc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:12.489260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180271eb-0d9e-4a6d-a940-c240a72a7edc None admin] PATH_INFO: `/v3/projects/53390afb4a404df79ab2d9ae31e8d058` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:12.490557 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180271eb-0d9e-4a6d-a940-c240a72a7edc None admin] RBAC: Authorizing `identity:get_project(project_id=53390afb4a404df79ab2d9ae31e8d058)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:12.494711 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180271eb-0d9e-4a6d-a940-c240a72a7edc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:12.494711 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 210/429] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:23:12 2022] GET /identity/v3/projects/53390afb4a404df79ab2d9ae31e8d058 => generated 298 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:12.500046 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9871a0-4013-499a-a794-4fc449ebb07f None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:12.616527 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9871a0-4013-499a-a794-4fc449ebb07f None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:12.654108 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32361dc9-f0d9-4a2f-b1d7-1cef02c224cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:12.655016 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32361dc9-f0d9-4a2f-b1d7-1cef02c224cd None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:12.655303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32361dc9-f0d9-4a2f-b1d7-1cef02c224cd None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:12.655585 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32361dc9-f0d9-4a2f-b1d7-1cef02c224cd None admin] PATH_INFO: `/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:12.656848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32361dc9-f0d9-4a2f-b1d7-1cef02c224cd None admin] RBAC: Authorizing `identity:get_group(group_id=53f45b0a49154be9a2c7124b7ee49fa7)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:12.658317 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32361dc9-f0d9-4a2f-b1d7-1cef02c224cd None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:12.660305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 220/430] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 22:23:12 2022] GET /identity/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:12.666106 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180271eb-0d9e-4a6d-a940-c240a72a7edc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:12.785890 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:12.817540 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180271eb-0d9e-4a6d-a940-c240a72a7edc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:12.858154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a898b1c-f97d-43a0-a2ce-d4793c9f7516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:12.858825 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a898b1c-f97d-43a0-a2ce-d4793c9f7516 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:12.860014 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a898b1c-f97d-43a0-a2ce-d4793c9f7516 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:12.860014 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a898b1c-f97d-43a0-a2ce-d4793c9f7516 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:12.860256 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a898b1c-f97d-43a0-a2ce-d4793c9f7516 None admin] RBAC: Adding query filter params (group.id=53f45b0a49154be9a2c7124b7ee49fa7, role.id=61397d4a1ccd4a8ca479f98186ed489b, scope.project.id=53390afb4a404df79ab2d9ae31e8d058) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:12.860563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a898b1c-f97d-43a0-a2ce-d4793c9f7516 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:12.861988 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a898b1c-f97d-43a0-a2ce-d4793c9f7516 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:12.869363 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 211/431] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 22:23:12 2022] GET /identity/v3/role_assignments?group.id=53f45b0a49154be9a2c7124b7ee49fa7&scope.project.id=53390afb4a404df79ab2d9ae31e8d058&role.id=61397d4a1ccd4a8ca479f98186ed489b => generated 611 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:13.002240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28cd3fd-b029-4020-b60b-d4e5d095ccd1 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:13.002632 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28cd3fd-b029-4020-b60b-d4e5d095ccd1 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:13.002812 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28cd3fd-b029-4020-b60b-d4e5d095ccd1 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:13.004234 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 221/432] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:13.010633 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b5f93-14d9-47c1-af82-c84b01e33b2a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:13.011156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b5f93-14d9-47c1-af82-c84b01e33b2a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:13.011230 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b5f93-14d9-47c1-af82-c84b01e33b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:13.032304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-019b5f93-14d9-47c1-af82-c84b01e33b2a None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:13.033537 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-019b5f93-14d9-47c1-af82-c84b01e33b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:13.126595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 212/433] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:13.153341 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28cd3fd-b029-4020-b60b-d4e5d095ccd1 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:13.171573 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:13.275807 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28cd3fd-b029-4020-b60b-d4e5d095ccd1 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:13.314314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d8fac-0dda-4579-a8ae-4747f1fd9ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:13.315163 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d8fac-0dda-4579-a8ae-4747f1fd9ccd None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:13.315453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d8fac-0dda-4579-a8ae-4747f1fd9ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:13.315736 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d8fac-0dda-4579-a8ae-4747f1fd9ccd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:13.327675 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d8fac-0dda-4579-a8ae-4747f1fd9ccd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:13.336309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-028d8fac-0dda-4579-a8ae-4747f1fd9ccd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:23:13.336309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:13.336309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:13.336309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:13.336309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:13.336309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:13.336309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:13.336309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:13.336309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:13.336309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:13.336309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:13.336309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:13.336309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:13.336309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:13.336309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:13.336309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:13.337210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:13.337210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:13.337210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:13.337210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:13.337210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:13.337210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:13.337210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:13.337210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:13.337210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:13.337210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:13.337210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:13.337210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:13.337210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:13.337210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:13.337210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:13.337210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:13.338088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:13.338088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:13.338088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:13.338088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:13.338088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:13.338088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:13.338088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:13.338088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:13.338088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:13.338088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:13.338088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 22:23:13.338088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:13.338088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 222/434] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:13.344152 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b5f93-14d9-47c1-af82-c84b01e33b2a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:13.439019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:13.474446 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b5f93-14d9-47c1-af82-c84b01e33b2a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:13.515242 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08f057-915d-4264-bca9-753ff10ffcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:13.515925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08f057-915d-4264-bca9-753ff10ffcfd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:13.516218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08f057-915d-4264-bca9-753ff10ffcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:13.516504 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08f057-915d-4264-bca9-753ff10ffcfd None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:13.517421 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08f057-915d-4264-bca9-753ff10ffcfd None admin] RBAC: Adding query filter params (name=member) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:13.517738 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08f057-915d-4264-bca9-753ff10ffcfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:13.519157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08f057-915d-4264-bca9-753ff10ffcfd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:13.519502 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d382e-5562-4850-b2c3-91c837e58d42 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:13.519980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d382e-5562-4850-b2c3-91c837e58d42 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:13.520483 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d382e-5562-4850-b2c3-91c837e58d42 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:13.522987 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 223/435] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:13.528031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e343a-1e9a-4715-bd21-236fb1a90142 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:13.528322 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e343a-1e9a-4715-bd21-236fb1a90142 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:13.528605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e343a-1e9a-4715-bd21-236fb1a90142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:13.529258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 213/436] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:23:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:13.535491 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08f057-915d-4264-bca9-753ff10ffcfd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:13.548867 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-434e343a-1e9a-4715-bd21-236fb1a90142 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:13.550423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-434e343a-1e9a-4715-bd21-236fb1a90142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:13.624231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 224/437] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:23:13.643871 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08f057-915d-4264-bca9-753ff10ffcfd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:13.647635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e343a-1e9a-4715-bd21-236fb1a90142 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:13.650610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:13.676887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:13.683035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a46b9-1d92-4340-8b35-6844fe815c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:13.683691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a46b9-1d92-4340-8b35-6844fe815c90 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:13.683973 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a46b9-1d92-4340-8b35-6844fe815c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:13.684251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a46b9-1d92-4340-8b35-6844fe815c90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:13.751538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e343a-1e9a-4715-bd21-236fb1a90142 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:13.758773 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a46b9-1d92-4340-8b35-6844fe815c90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:13.760285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a46b9-1d92-4340-8b35-6844fe815c90 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:13.784397 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccca10c-f247-42f2-a7a8-41e4593467d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:13.785201 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccca10c-f247-42f2-a7a8-41e4593467d8 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:13.785483 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccca10c-f247-42f2-a7a8-41e4593467d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:13.785757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccca10c-f247-42f2-a7a8-41e4593467d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:13.796402 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccca10c-f247-42f2-a7a8-41e4593467d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:13.804469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-7ccca10c-f247-42f2-a7a8-41e4593467d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:23:13.804469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:13.804469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:13.804469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:13.804469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:13.804469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:13.804469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:13.804469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:13.804469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:13.804469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:13.804469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:13.804469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:13.804469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:13.804469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:13.804469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:13.804469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:13.805353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:13.805353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:13.805353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:13.805353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:13.805353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:13.805353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:13.805353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:13.805353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:13.805353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:13.805353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:13.805353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:13.805353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:13.805353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:13.805353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:13.805353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:13.805353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:13.806144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:13.806144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:13.806144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:13.806144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:13.806144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:13.806144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:13.806144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:13.806144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:13.806144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:13.806144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:13.806144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:23:13.806144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:13.806144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 225/438] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:23:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:13.806827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db018bee-69e0-4a1a-ae4a-985810ef5f8e None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:13.807134 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db018bee-69e0-4a1a-ae4a-985810ef5f8e None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:13.807417 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db018bee-69e0-4a1a-ae4a-985810ef5f8e None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:13.808668 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 226/439] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:13.811276 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db018bee-69e0-4a1a-ae4a-985810ef5f8e None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:13.866054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 214/440] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 22:23:13.867366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595c4ea-730b-4f07-b3ad-af359480af2f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:13.867662 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595c4ea-730b-4f07-b3ad-af359480af2f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:13.867942 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595c4ea-730b-4f07-b3ad-af359480af2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:13.874053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:13.885466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4595c4ea-730b-4f07-b3ad-af359480af2f None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:13.886653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4595c4ea-730b-4f07-b3ad-af359480af2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:13.911908 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:13.913260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db018bee-69e0-4a1a-ae4a-985810ef5f8e None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:13.946063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0ce5d-4a88-49c5-90b7-075f9532839c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:13.946722 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0ce5d-4a88-49c5-90b7-075f9532839c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:13.947039 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0ce5d-4a88-49c5-90b7-075f9532839c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:13.947320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0ce5d-4a88-49c5-90b7-075f9532839c None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:13.948063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0ce5d-4a88-49c5-90b7-075f9532839c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:13.948374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0ce5d-4a88-49c5-90b7-075f9532839c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:13.949737 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0ce5d-4a88-49c5-90b7-075f9532839c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:13.952512 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 215/441] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:13.953341 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595c4ea-730b-4f07-b3ad-af359480af2f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:13.965173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 227/442] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:13.970374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0ce5d-4a88-49c5-90b7-075f9532839c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:14.052796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595c4ea-730b-4f07-b3ad-af359480af2f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:14.070491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0ce5d-4a88-49c5-90b7-075f9532839c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:14.087434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891dab63-f3cf-4f6c-8695-ad1cef36baac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:14.088285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dab63-f3cf-4f6c-8695-ad1cef36baac None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:14.088570 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dab63-f3cf-4f6c-8695-ad1cef36baac None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:14.088849 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dab63-f3cf-4f6c-8695-ad1cef36baac None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:14.090017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891dab63-f3cf-4f6c-8695-ad1cef36baac None admin] RBAC: Authorizing `identity:get_project(project_id=6525e5222e3342a3a26a6369518c7577)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:14.091493 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891dab63-f3cf-4f6c-8695-ad1cef36baac None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:14.093164 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 216/443] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:23:13 2022] GET /identity/v3/projects/6525e5222e3342a3a26a6369518c7577 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:14.093916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891dab63-f3cf-4f6c-8695-ad1cef36baac None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:14.096033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:14.106334 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1df5c-2882-4c01-9981-beb6e62d9106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:14.107027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1df5c-2882-4c01-9981-beb6e62d9106 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:14.107326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1df5c-2882-4c01-9981-beb6e62d9106 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:14.107614 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1df5c-2882-4c01-9981-beb6e62d9106 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:14.130979 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:14.177652 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1df5c-2882-4c01-9981-beb6e62d9106 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:14.179199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1df5c-2882-4c01-9981-beb6e62d9106 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:14.200655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891dab63-f3cf-4f6c-8695-ad1cef36baac None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:14.233613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7435dd5-d084-4c36-a09e-6cd499a30481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:14.234402 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7435dd5-d084-4c36-a09e-6cd499a30481 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:14.234690 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7435dd5-d084-4c36-a09e-6cd499a30481 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:14.235059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7435dd5-d084-4c36-a09e-6cd499a30481 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:14.245329 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7435dd5-d084-4c36-a09e-6cd499a30481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:14.252831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c7435dd5-d084-4c36-a09e-6cd499a30481 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:23:14.252831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:14.252831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:14.252831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:14.252831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:14.252831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:14.252831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:14.252831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:14.252831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:14.252831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:14.252831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:14.252831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:14.252831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:14.252831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:14.252831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:14.252831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:14.253875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:14.253875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:14.253875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:14.253875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:14.253875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:14.253875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:14.253875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:14.253875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:14.253875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:14.253875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:14.253875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:14.253875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:14.253875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:14.253875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:14.253875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:14.253875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:14.255206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:14.255206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:14.255206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:14.255206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:14.255206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:14.255206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:14.255206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:14.255206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:14.255206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:14.255206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:14.255206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:23:14.255206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:14.255206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 217/444] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:23:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:14.255206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7435dd5-d084-4c36-a09e-6cd499a30481 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:14.282967 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 228/445] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 22:23:14.282967 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1df5c-2882-4c01-9981-beb6e62d9106 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:14.307828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:14.345893 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:14.356399 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7435dd5-d084-4c36-a09e-6cd499a30481 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:14.378342 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1df5c-2882-4c01-9981-beb6e62d9106 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:14.389184 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b99585-02f1-4828-8952-c7c7c6e3c46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:14.390043 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b99585-02f1-4828-8952-c7c7c6e3c46c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:14.390332 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b99585-02f1-4828-8952-c7c7c6e3c46c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:14.390620 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b99585-02f1-4828-8952-c7c7c6e3c46c None admin] PATH_INFO: `/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:14.391837 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b99585-02f1-4828-8952-c7c7c6e3c46c None admin] RBAC: Authorizing `identity:get_group(group_id=53f45b0a49154be9a2c7124b7ee49fa7)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:14.393284 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b99585-02f1-4828-8952-c7c7c6e3c46c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:14.394891 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 218/446] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 22:23:14 2022] GET /identity/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:14.395691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b99585-02f1-4828-8952-c7c7c6e3c46c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:14.415344 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef307a-c01e-49e6-8a5b-c3cc3975aa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:14.416010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef307a-c01e-49e6-8a5b-c3cc3975aa9f None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:14.416299 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef307a-c01e-49e6-8a5b-c3cc3975aa9f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:14.416578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef307a-c01e-49e6-8a5b-c3cc3975aa9f None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:14.417387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef307a-c01e-49e6-8a5b-c3cc3975aa9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:14.417699 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef307a-c01e-49e6-8a5b-c3cc3975aa9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:14.419101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef307a-c01e-49e6-8a5b-c3cc3975aa9f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:14.427469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 229/447] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:14.428931 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef307a-c01e-49e6-8a5b-c3cc3975aa9f None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:14.493519 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b99585-02f1-4828-8952-c7c7c6e3c46c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:14.515767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:14.528516 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef307a-c01e-49e6-8a5b-c3cc3975aa9f None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:14.529808 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aec7dc-7bb6-4553-9699-d1a35fa74018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:14.530661 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aec7dc-7bb6-4553-9699-d1a35fa74018 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:14.530970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aec7dc-7bb6-4553-9699-d1a35fa74018 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:14.531252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aec7dc-7bb6-4553-9699-d1a35fa74018 None admin] PATH_INFO: `/v3/projects/6525e5222e3342a3a26a6369518c7577` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:14.532430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aec7dc-7bb6-4553-9699-d1a35fa74018 None admin] RBAC: Authorizing `identity:get_project(project_id=6525e5222e3342a3a26a6369518c7577)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:14.533862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aec7dc-7bb6-4553-9699-d1a35fa74018 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:14.535562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 219/448] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:23:14 2022] GET /identity/v3/projects/6525e5222e3342a3a26a6369518c7577 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:14.536311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aec7dc-7bb6-4553-9699-d1a35fa74018 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:14.563054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:14.567222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85399a0-8663-42c2-9443-b1c66b7553e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:14.567977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85399a0-8663-42c2-9443-b1c66b7553e8 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:14.568297 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85399a0-8663-42c2-9443-b1c66b7553e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:14.568583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85399a0-8663-42c2-9443-b1c66b7553e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:14.570129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85399a0-8663-42c2-9443-b1c66b7553e8 None admin] RBAC: Adding query filter params (group.id=53f45b0a49154be9a2c7124b7ee49fa7, role.id=1baba2239751440eb25af0d0fd2d072d, scope.project.id=6525e5222e3342a3a26a6369518c7577) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:14.570451 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85399a0-8663-42c2-9443-b1c66b7553e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:14.571937 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85399a0-8663-42c2-9443-b1c66b7553e8 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:14.577780 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 230/449] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 22:23:14 2022] GET /identity/v3/role_assignments?group.id=53f45b0a49154be9a2c7124b7ee49fa7&scope.project.id=6525e5222e3342a3a26a6369518c7577&role.id=1baba2239751440eb25af0d0fd2d072d => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:14.578570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85399a0-8663-42c2-9443-b1c66b7553e8 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:14.668182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aec7dc-7bb6-4553-9699-d1a35fa74018 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:14.689412 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85399a0-8663-42c2-9443-b1c66b7553e8 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:14.707987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7135abe-bb65-42cb-9685-11d7e53cf772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:14.709268 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7135abe-bb65-42cb-9685-11d7e53cf772 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:14.710703 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7135abe-bb65-42cb-9685-11d7e53cf772 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:14.711205 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7135abe-bb65-42cb-9685-11d7e53cf772 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:14.730520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c7b04-50fa-4d01-85e8-cf77147b5b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:14.731416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c7b04-50fa-4d01-85e8-cf77147b5b76 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:14.731706 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c7b04-50fa-4d01-85e8-cf77147b5b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:14.731988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c7b04-50fa-4d01-85e8-cf77147b5b76 None admin] PATH_INFO: `/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:14.733348 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c7b04-50fa-4d01-85e8-cf77147b5b76 None admin] RBAC: Authorizing `identity:get_group(group_id=53f45b0a49154be9a2c7124b7ee49fa7)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:14.734792 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c7b04-50fa-4d01-85e8-cf77147b5b76 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:14.736446 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 231/450] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 22:23:14 2022] GET /identity/v3/groups/53f45b0a49154be9a2c7124b7ee49fa7 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:14.742992 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c7b04-50fa-4d01-85e8-cf77147b5b76 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:14.770092 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:14.796831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7135abe-bb65-42cb-9685-11d7e53cf772 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:14.798452 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7135abe-bb65-42cb-9685-11d7e53cf772 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:14.831185 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:14.863475 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c7b04-50fa-4d01-85e8-cf77147b5b76 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:14.904204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ea56e-fe8d-4b0a-9c63-517ec36e4f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:14.904862 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ea56e-fe8d-4b0a-9c63-517ec36e4f1f None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:14.905143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ea56e-fe8d-4b0a-9c63-517ec36e4f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:14.905417 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ea56e-fe8d-4b0a-9c63-517ec36e4f1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:14.906356 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ea56e-fe8d-4b0a-9c63-517ec36e4f1f None admin] RBAC: Adding query filter params (group.id=53f45b0a49154be9a2c7124b7ee49fa7, role.id=61397d4a1ccd4a8ca479f98186ed489b, scope.project.id=6525e5222e3342a3a26a6369518c7577) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:14.906674 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ea56e-fe8d-4b0a-9c63-517ec36e4f1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:14.908138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ea56e-fe8d-4b0a-9c63-517ec36e4f1f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:14.918973 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 232/451] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 22:23:14 2022] GET /identity/v3/role_assignments?group.id=53f45b0a49154be9a2c7124b7ee49fa7&scope.project.id=6525e5222e3342a3a26a6369518c7577&role.id=61397d4a1ccd4a8ca479f98186ed489b => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:14.926213 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 220/452] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:14.927800 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ea56e-fe8d-4b0a-9c63-517ec36e4f1f None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:15.019935 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:15.036599 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ea56e-fe8d-4b0a-9c63-517ec36e4f1f None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:15.072701 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27708b-56d3-41b3-9de2-1c12857dfb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:15.073594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27708b-56d3-41b3-9de2-1c12857dfb3c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:15.073881 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27708b-56d3-41b3-9de2-1c12857dfb3c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:15.074162 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27708b-56d3-41b3-9de2-1c12857dfb3c None admin] PATH_INFO: `/v3/projects/1aef1dab34da4006bf2529ee1b5120da` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:15.075367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27708b-56d3-41b3-9de2-1c12857dfb3c None admin] RBAC: Authorizing `identity:get_project(project_id=1aef1dab34da4006bf2529ee1b5120da)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:15.076814 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27708b-56d3-41b3-9de2-1c12857dfb3c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:15.078642 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 233/453] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:23:14 2022] GET /identity/v3/projects/1aef1dab34da4006bf2529ee1b5120da => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:15.084121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7135abe-bb65-42cb-9685-11d7e53cf772 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:15.191302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7135abe-bb65-42cb-9685-11d7e53cf772 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:15.218796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:15.228801 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e0243-4c11-4b65-9569-fb6891576ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:15.229666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e0243-4c11-4b65-9569-fb6891576ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:15.229957 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e0243-4c11-4b65-9569-fb6891576ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:15.230241 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e0243-4c11-4b65-9569-fb6891576ab7 None admin] PATH_INFO: `/v3/groups/d888fe884c8443b2961ca5e7fe664e7a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:15.231456 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e0243-4c11-4b65-9569-fb6891576ab7 None admin] RBAC: Authorizing `identity:get_group(group_id=d888fe884c8443b2961ca5e7fe664e7a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:15.232897 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e0243-4c11-4b65-9569-fb6891576ab7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:15.234525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 221/454] 10.222.0.43 () {56 vars in 1180 bytes} [Thu Jun 23 22:23:15 2022] GET /identity/v3/groups/d888fe884c8443b2961ca5e7fe664e7a => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:15.240498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27708b-56d3-41b3-9de2-1c12857dfb3c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:15.339526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27708b-56d3-41b3-9de2-1c12857dfb3c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:15.371733 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1f00e-33c9-4ab5-a753-705658f6a926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:15.372383 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1f00e-33c9-4ab5-a753-705658f6a926 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:15.372676 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1f00e-33c9-4ab5-a753-705658f6a926 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:15.372963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1f00e-33c9-4ab5-a753-705658f6a926 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:15.373828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1f00e-33c9-4ab5-a753-705658f6a926 None admin] RBAC: Adding query filter params (group.id=d888fe884c8443b2961ca5e7fe664e7a, role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.project.id=1aef1dab34da4006bf2529ee1b5120da) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:15.374148 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1f00e-33c9-4ab5-a753-705658f6a926 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:15.375630 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1f00e-33c9-4ab5-a753-705658f6a926 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:15.381886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 234/455] 10.222.0.43 () {56 vars in 1376 bytes} [Thu Jun 23 22:23:15 2022] GET /identity/v3/role_assignments?group.id=d888fe884c8443b2961ca5e7fe664e7a&scope.project.id=1aef1dab34da4006bf2529ee1b5120da&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:17.797954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfff05-f4e5-4a6a-a214-fa92e6077806 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:17.798838 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfff05-f4e5-4a6a-a214-fa92e6077806 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:17.799294 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfff05-f4e5-4a6a-a214-fa92e6077806 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:17.801032 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 222/456] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:17.806935 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bd024-6a7c-42c5-a801-be4486c98e8d None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:17.807732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bd024-6a7c-42c5-a801-be4486c98e8d None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:17.808172 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bd024-6a7c-42c5-a801-be4486c98e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:17.831962 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-1c8bd024-6a7c-42c5-a801-be4486c98e8d None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:17.833977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1c8bd024-6a7c-42c5-a801-be4486c98e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:17.856347 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:17.929678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 235/457] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:17.965532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfff05-f4e5-4a6a-a214-fa92e6077806 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:18.039313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbaaff-0d6a-4a5e-86be-4a6ec75f21f2 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:18.039636 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbaaff-0d6a-4a5e-86be-4a6ec75f21f2 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:18.039884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dbaaff-0d6a-4a5e-86be-4a6ec75f21f2 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:18.041266 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 236/458] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:18.046660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e44bc-2660-4b63-957c-644baae3fccf None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:18.046970 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e44bc-2660-4b63-957c-644baae3fccf None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:18.047261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e44bc-2660-4b63-957c-644baae3fccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:18.080473 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-652e44bc-2660-4b63-957c-644baae3fccf None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:18.081600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-652e44bc-2660-4b63-957c-644baae3fccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:18.095663 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfff05-f4e5-4a6a-a214-fa92e6077806 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:18.179145 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b63f1d-379a-4986-81f9-391bb33d17b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:18.179955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b63f1d-379a-4986-81f9-391bb33d17b9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:18.180243 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b63f1d-379a-4986-81f9-391bb33d17b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:18.180527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b63f1d-379a-4986-81f9-391bb33d17b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:18.201932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b63f1d-379a-4986-81f9-391bb33d17b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:18.203447 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b63f1d-379a-4986-81f9-391bb33d17b9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:18.209588 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 237/459] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:23:18.234916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-21b63f1d-379a-4986-81f9-391bb33d17b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:18.234916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:18.234916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:18.234916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:18.234916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:18.234916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:18.234916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:18.234916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:18.234916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:18.234916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:18.234916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:18.234916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:18.234916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:18.234916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:18.234916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:18.235877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:18.235877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:18.235877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:18.235877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:18.235877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:18.235877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:18.235877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:18.235877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:18.235877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:18.235877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:18.235877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:18.235877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:18.235877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:18.235877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:18.235877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:18.235877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:18.235877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:18.236740 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:18.236740 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:18.236740 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:18.236740 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:18.236740 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 223/460] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:18.242746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e44bc-2660-4b63-957c-644baae3fccf None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:18.281087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b63f1d-379a-4986-81f9-391bb33d17b9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:18.334242 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:18.371513 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:18.393904 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b63f1d-379a-4986-81f9-391bb33d17b9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:18.403529 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e44bc-2660-4b63-957c-644baae3fccf None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:18.440298 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add204b-82c6-4fce-a3c6-34d21d08d3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:18.440830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab2630-e5ec-473f-9a56-997c1f2ab2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:18.441539 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add204b-82c6-4fce-a3c6-34d21d08d3ea None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:18.441665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab2630-e5ec-473f-9a56-997c1f2ab2cc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:18.441926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab2630-e5ec-473f-9a56-997c1f2ab2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:18.442199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab2630-e5ec-473f-9a56-997c1f2ab2cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:18.442556 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add204b-82c6-4fce-a3c6-34d21d08d3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:18.443060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add204b-82c6-4fce-a3c6-34d21d08d3ea None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:18.444603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add204b-82c6-4fce-a3c6-34d21d08d3ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:18.445120 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add204b-82c6-4fce-a3c6-34d21d08d3ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:18.446697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add204b-82c6-4fce-a3c6-34d21d08d3ea None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:18.454051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab2630-e5ec-473f-9a56-997c1f2ab2cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:18.455552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab2630-e5ec-473f-9a56-997c1f2ab2cc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:18.465899 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 238/461] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:18.467609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba5259-8c9c-4bdc-8c00-5ef534f764a2 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:18.468101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba5259-8c9c-4bdc-8c00-5ef534f764a2 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:18.468577 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba5259-8c9c-4bdc-8c00-5ef534f764a2 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:18.474090 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 239/462] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:18.475213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba5259-8c9c-4bdc-8c00-5ef534f764a2 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:18.481544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-95ab2630-e5ec-473f-9a56-997c1f2ab2cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:18.481544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:18.481544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:18.481544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:18.481544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:18.481544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:18.481544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:18.481544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:18.481544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:18.481544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:18.481544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:18.481544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:18.481544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:18.481544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:18.481544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:18.482373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:18.482373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:18.482373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:18.482373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:18.482373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:18.482373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:18.482373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:18.482373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:18.482373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:18.482373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:18.482373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:18.482373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:18.482373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:18.482373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:18.482373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:18.482373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:18.482373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:18.483267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:18.483267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:18.483267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:18.483267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:18.487551 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 224/463] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:23:18.489293 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cef3f-75a3-4735-b9f2-087d282c2984 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:18.489590 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cef3f-75a3-4735-b9f2-087d282c2984 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:18.489880 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cef3f-75a3-4735-b9f2-087d282c2984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:18.511634 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-613cef3f-75a3-4735-b9f2-087d282c2984 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:18.512865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-613cef3f-75a3-4735-b9f2-087d282c2984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:18.595404 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba5259-8c9c-4bdc-8c00-5ef534f764a2 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:18.602942 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 225/464] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:18.602942 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613cef3f-75a3-4735-b9f2-087d282c2984 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:18.632794 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:18.648633 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6ef9d-1853-4834-b11e-0dd889dd5592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:18.649162 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6ef9d-1853-4834-b11e-0dd889dd5592 None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:18.649451 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6ef9d-1853-4834-b11e-0dd889dd5592 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:18.649732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6ef9d-1853-4834-b11e-0dd889dd5592 None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:18.650439 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6ef9d-1853-4834-b11e-0dd889dd5592 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:18.651935 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6ef9d-1853-4834-b11e-0dd889dd5592 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:18.660243 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:18.677175 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 240/465] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 22:23:18 2022] POST /identity/v3/users => generated 246 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:23:18.678811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdea28-8358-4039-95b6-c0a73b867dec None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:18.679324 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdea28-8358-4039-95b6-c0a73b867dec None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:18.679807 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdea28-8358-4039-95b6-c0a73b867dec None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:18.681633 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 241/466] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:18.682642 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affdea28-8358-4039-95b6-c0a73b867dec None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:18.745038 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613cef3f-75a3-4735-b9f2-087d282c2984 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:18.786965 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003540b7-5389-4bd2-ba03-01b9bb201c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:18.789366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003540b7-5389-4bd2-ba03-01b9bb201c13 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:18.789868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003540b7-5389-4bd2-ba03-01b9bb201c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:18.790391 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003540b7-5389-4bd2-ba03-01b9bb201c13 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:18.791475 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003540b7-5389-4bd2-ba03-01b9bb201c13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:18.792032 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003540b7-5389-4bd2-ba03-01b9bb201c13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:18.793696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003540b7-5389-4bd2-ba03-01b9bb201c13 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:18.808543 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 226/467] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:18.810659 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edc3aa-fef5-454b-8a76-c8133b90a40a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:18.811236 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edc3aa-fef5-454b-8a76-c8133b90a40a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:18.811757 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edc3aa-fef5-454b-8a76-c8133b90a40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:18.832379 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-67edc3aa-fef5-454b-8a76-c8133b90a40a None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:18.833925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-67edc3aa-fef5-454b-8a76-c8133b90a40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:18.835599 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affdea28-8358-4039-95b6-c0a73b867dec None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:18.879261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a63f16-3dda-4bde-9231-ed5e32a3bb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:18.880276 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a63f16-3dda-4bde-9231-ed5e32a3bb96 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:18.880276 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a63f16-3dda-4bde-9231-ed5e32a3bb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:18.880526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a63f16-3dda-4bde-9231-ed5e32a3bb96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:18.896579 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a63f16-3dda-4bde-9231-ed5e32a3bb96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:18.898598 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a63f16-3dda-4bde-9231-ed5e32a3bb96 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:18.919246 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 227/468] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:18.920390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edc3aa-fef5-454b-8a76-c8133b90a40a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:18.920770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-67a63f16-3dda-4bde-9231-ed5e32a3bb96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:18.920770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:18.920770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:18.920770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:18.920770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:18.920770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:18.920770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:18.920770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:18.920770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:18.920770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:18.920770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:18.920770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:18.920770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:18.920770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:18.920770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:18.921600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:18.921600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:18.921600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:18.921600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:18.921600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:18.921600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:18.921600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:18.921600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:18.921600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:18.921600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:18.921600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:18.921600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:18.921600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:18.921600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:18.921600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:18.921600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:18.921600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:18.922442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:18.922442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:18.922442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:18.922442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:18.922442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 242/469] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:18.927960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a63f16-3dda-4bde-9231-ed5e32a3bb96 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:18.953170 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:18.979562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:19.052704 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edc3aa-fef5-454b-8a76-c8133b90a40a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:19.056608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a63f16-3dda-4bde-9231-ed5e32a3bb96 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:19.096056 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943d144-88b0-4442-b67b-f014020bbf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:19.097047 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943d144-88b0-4442-b67b-f014020bbf2a None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:19.097494 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943d144-88b0-4442-b67b-f014020bbf2a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:19.097928 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943d144-88b0-4442-b67b-f014020bbf2a None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:19.098809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943d144-88b0-4442-b67b-f014020bbf2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:19.098913 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba68268-abde-43fc-9aae-757dc6d8e39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:19.099603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba68268-abde-43fc-9aae-757dc6d8e39a None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:19.099899 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba68268-abde-43fc-9aae-757dc6d8e39a None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:19.100811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba68268-abde-43fc-9aae-757dc6d8e39a None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:19.101502 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943d144-88b0-4442-b67b-f014020bbf2a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:19.101606 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba68268-abde-43fc-9aae-757dc6d8e39a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:19.101861 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba68268-abde-43fc-9aae-757dc6d8e39a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:19.103347 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba68268-abde-43fc-9aae-757dc6d8e39a None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:19.118549 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 243/470] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:19.119614 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba68268-abde-43fc-9aae-757dc6d8e39a None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:19.125320 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 228/471] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 22:23:18 2022] POST /identity/v3/users => generated 248 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:23:19.126059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943d144-88b0-4442-b67b-f014020bbf2a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:19.239755 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:19.259921 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba68268-abde-43fc-9aae-757dc6d8e39a None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:19.263719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:19.279461 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943d144-88b0-4442-b67b-f014020bbf2a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:19.305293 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d434ae-7e8b-40f5-be4d-74ac483b37f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:19.306362 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d434ae-7e8b-40f5-be4d-74ac483b37f9 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:19.306941 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d434ae-7e8b-40f5-be4d-74ac483b37f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:19.307343 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d434ae-7e8b-40f5-be4d-74ac483b37f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:19.321130 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d434ae-7e8b-40f5-be4d-74ac483b37f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:19.322844 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d434ae-7e8b-40f5-be4d-74ac483b37f9 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:19.329331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fab838-7fbf-47ac-9537-d26bcb5a73e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:19.330036 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fab838-7fbf-47ac-9537-d26bcb5a73e4 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:19.330329 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fab838-7fbf-47ac-9537-d26bcb5a73e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:19.330612 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fab838-7fbf-47ac-9537-d26bcb5a73e4 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:19.332039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fab838-7fbf-47ac-9537-d26bcb5a73e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:19.334017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fab838-7fbf-47ac-9537-d26bcb5a73e4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:19.346884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-e7d434ae-7e8b-40f5-be4d-74ac483b37f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:19.346884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:19.346884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:19.346884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:19.346884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:19.346884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:19.346884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:19.346884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:19.346884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:19.346884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:19.346884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:19.346884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:19.346884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:19.346884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:19.346884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:19.348195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:19.348195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:19.348195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:19.348195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:19.348195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:19.348195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:19.348195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:19.348195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:19.348195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:19.348195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:19.348195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:19.348195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:19.348195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:19.348195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:19.348195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:19.348195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:19.348195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:19.349573 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:19.349573 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:19.349573 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:19.349573 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:19.349573 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 244/472] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:19.350562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b58d3b-4628-49eb-973c-131a170bea80 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:19.351375 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b58d3b-4628-49eb-973c-131a170bea80 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:19.351830 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b58d3b-4628-49eb-973c-131a170bea80 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:19.356671 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 245/473] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:19.357727 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b58d3b-4628-49eb-973c-131a170bea80 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:19.360319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 229/474] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 22:23:19 2022] POST /identity/v3/users => generated 248 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:23:19.361732 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617c66d-dd49-470b-887b-201c8b147162 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:19.362387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617c66d-dd49-470b-887b-201c8b147162 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:19.362750 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617c66d-dd49-470b-887b-201c8b147162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:19.387266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e617c66d-dd49-470b-887b-201c8b147162 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:19.388996 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e617c66d-dd49-470b-887b-201c8b147162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:19.486418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 230/475] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:19.503325 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b58d3b-4628-49eb-973c-131a170bea80 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:19.516069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617c66d-dd49-470b-887b-201c8b147162 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:19.542570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54097d1f-0f88-42dd-9a75-13ec3537647c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:19.543331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54097d1f-0f88-42dd-9a75-13ec3537647c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:19.543670 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54097d1f-0f88-42dd-9a75-13ec3537647c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:19.543953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54097d1f-0f88-42dd-9a75-13ec3537647c None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:19.544679 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54097d1f-0f88-42dd-9a75-13ec3537647c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:19.544986 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54097d1f-0f88-42dd-9a75-13ec3537647c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:19.546498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54097d1f-0f88-42dd-9a75-13ec3537647c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:19.559688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 246/476] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:19.564745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54097d1f-0f88-42dd-9a75-13ec3537647c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:19.586530 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:19.586792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:19.639140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617c66d-dd49-470b-887b-201c8b147162 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:19.692772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb24e0-ac96-4ae5-9337-82225240ed1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:19.694142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb24e0-ac96-4ae5-9337-82225240ed1d None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:19.694904 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb24e0-ac96-4ae5-9337-82225240ed1d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:19.695352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb24e0-ac96-4ae5-9337-82225240ed1d None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:19.696490 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb24e0-ac96-4ae5-9337-82225240ed1d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:19.698651 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb24e0-ac96-4ae5-9337-82225240ed1d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:19.700465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54097d1f-0f88-42dd-9a75-13ec3537647c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:19.707963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.sql.core [None req-41eb24e0-ac96-4ae5-9337-82225240ed1d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 22:23:19.707963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 22:23:19.707963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [parameters: {'id': '5ce18c4e9ea84cf3a834a049fae7fb1a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 22:23:19.707963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80203) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 22:23:19.731466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-41eb24e0-ac96-4ae5-9337-82225240ed1d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 22:23:19.731466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:19.731466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 22:23:19.731466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 22:23:19.731466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 22:23:19.731466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 22:23:19.731466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 22:23:19.731466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 22:23:19.731466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 22:23:19.731466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) Jun 23 22:23:19.731466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 22:23:19.731466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 22:23:19.731466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 22:23:19.731466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() Jun 23 22:23:19.731466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 22:23:19.732544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 22:23:19.732544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 22:23:19.732544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 22:23:19.732544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 22:23:19.732544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 22:23:19.732544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 22:23:19.732544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 22:23:19.732544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 22:23:19.732544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:19.732544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 22:23:19.732544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:19.732544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:19.732544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 22:23:19.732544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 22:23:19.732544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 23 22:23:19.732544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return ref.to_dict() Jun 23 22:23:19.732544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 22:23:19.732544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) Jun 23 22:23:19.733574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 22:23:19.733574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application yield resource Jun 23 22:23:19.733574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 22:23:19.733574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) Jun 23 22:23:19.733574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 22:23:19.733574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 22:23:19.733574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 22:23:19.733574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 22:23:19.733574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 22:23:19.733574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise self.value Jun 23 22:23:19.733574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 22:23:19.733574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 22:23:19.733574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 22:23:19.733574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application session.commit() Jun 23 22:23:19.733574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 22:23:19.733574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 22:23:19.733574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 22:23:19.735384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 22:23:19.735384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 22:23:19.735384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.flush() Jun 23 22:23:19.735384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 22:23:19.735384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 22:23:19.735384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 22:23:19.735384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 22:23:19.735384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 22:23:19.735384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application compat.raise_( Jun 23 22:23:19.735384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 22:23:19.735384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception Jun 23 22:23:19.735384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 22:23:19.735384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 22:23:19.735384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 22:23:19.735384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 22:23:19.735384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 22:23:19.735384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 22:23:19.736616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 22:23:19.736616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 22:23:19.736616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 22:23:19.736616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 22:23:19.736616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 22:23:19.736616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 22:23:19.736616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 22:23:19.736616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 22:23:19.736616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 22:23:19.736616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 22:23:19.736616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 22:23:19.736616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 22:23:19.736616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 22:23:19.736616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 22:23:19.736616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 22:23:19.736616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception Jun 23 22:23:19.737602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 22:23:19.737602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 22:23:19.737602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 22:23:19.737602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 22:23:19.737602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 22:23:19.737602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 22:23:19.737602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 22:23:19.737602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) Jun 23 22:23:19.737602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 22:23:19.737602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 22:23:19.737602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 22:23:19.737602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() Jun 23 22:23:19.737602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 22:23:19.737602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 22:23:19.737602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 22:23:19.737602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 22:23:19.738658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 22:23:19.738658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 22:23:19.738658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 22:23:19.738658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 22:23:19.738658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 22:23:19.738658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 22:23:19.738658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [parameters: {'id': '5ce18c4e9ea84cf3a834a049fae7fb1a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 22:23:19.738658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 22:23:19.738658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:19.738658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:23:19.738658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:19.738658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:19.738658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:19.738658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:19.738658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:19.738658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:19.739672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:19.739672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:19.739672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:19.739672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:19.739672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:19.739672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:19.739672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 23 22:23:19.739672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 23 22:23:19.739672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:19.739672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:19.739672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 23 22:23:19.739672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 23 22:23:19.739672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 22:23:19.739672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 22:23:19.739672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 22:23:19.739672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:19.741214 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 231/477] 10.222.0.43 () {60 vars in 1131 bytes} [Thu Jun 23 22:23:19 2022] POST /identity/v3/roles => generated 144 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:19.742003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb24e0-ac96-4ae5-9337-82225240ed1d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:19.751014 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7d3d5-2b15-47a5-84c9-d4899b9353ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:19.751733 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7d3d5-2b15-47a5-84c9-d4899b9353ee None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:19.752095 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7d3d5-2b15-47a5-84c9-d4899b9353ee None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:19.752495 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7d3d5-2b15-47a5-84c9-d4899b9353ee None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:19.753391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7d3d5-2b15-47a5-84c9-d4899b9353ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:19.755086 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7d3d5-2b15-47a5-84c9-d4899b9353ee None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:19.782070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 247/478] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 22:23:19 2022] POST /identity/v3/users => generated 249 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:23:19.888722 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb24e0-ac96-4ae5-9337-82225240ed1d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:19.932580 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:19.950242 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b0d9e-8bb9-4c78-a790-96c1f5eafa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:19.951198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b0d9e-8bb9-4c78-a790-96c1f5eafa2f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:19.951568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b0d9e-8bb9-4c78-a790-96c1f5eafa2f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:19.951926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b0d9e-8bb9-4c78-a790-96c1f5eafa2f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:19.966666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b0d9e-8bb9-4c78-a790-96c1f5eafa2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:19.975868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d31b0d9e-8bb9-4c78-a790-96c1f5eafa2f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:23:19.975868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:19.975868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:19.975868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:19.975868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:19.975868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:19.975868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:19.975868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:19.975868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:19.975868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:19.975868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:19.975868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:19.975868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:19.975868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:19.975868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:19.975868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:19.977287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:19.977287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:19.977287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:19.977287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:19.977287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:19.977287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:19.977287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:19.977287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:19.977287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:19.977287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:19.977287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:19.977287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:19.977287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:19.977287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:19.977287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:19.977287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:19.978264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:19.978264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:19.978264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:19.978264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:19.978264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:19.978264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:19.978264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:19.978264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:19.978264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:19.978264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:19.978264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 22:23:19.978264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:19.979685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 232/479] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:23:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:19.987993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7d3d5-2b15-47a5-84c9-d4899b9353ee None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:20.064854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:20.097679 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7d3d5-2b15-47a5-84c9-d4899b9353ee None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:20.153151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851d692-82e1-4372-aca7-43090986b97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:20.154087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851d692-82e1-4372-aca7-43090986b97e None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:20.154555 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851d692-82e1-4372-aca7-43090986b97e None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:20.155007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851d692-82e1-4372-aca7-43090986b97e None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:20.155920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851d692-82e1-4372-aca7-43090986b97e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:20.156382 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851d692-82e1-4372-aca7-43090986b97e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:20.157930 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851d692-82e1-4372-aca7-43090986b97e None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:20.168366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 248/480] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:21.326062 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036a35f-98ed-497b-b252-7acbf728b8f0 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:21.326914 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036a35f-98ed-497b-b252-7acbf728b8f0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:21.327375 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1036a35f-98ed-497b-b252-7acbf728b8f0 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:21.328959 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 233/481] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:21.345307 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f181530-5ce7-4195-ada0-47d871d45dc3 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:21.345794 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f181530-5ce7-4195-ada0-47d871d45dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:21.346067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f181530-5ce7-4195-ada0-47d871d45dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:21.369015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-3f181530-5ce7-4195-ada0-47d871d45dc3 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:21.370243 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3f181530-5ce7-4195-ada0-47d871d45dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:21.456283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 249/482] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:21.480760 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036a35f-98ed-497b-b252-7acbf728b8f0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:21.624210 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1036a35f-98ed-497b-b252-7acbf728b8f0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:21.675668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea4e33-bb68-44ad-9235-64832725d0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:21.676470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea4e33-bb68-44ad-9235-64832725d0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:21.676755 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea4e33-bb68-44ad-9235-64832725d0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:21.677030 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea4e33-bb68-44ad-9235-64832725d0e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:21.689137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea4e33-bb68-44ad-9235-64832725d0e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:21.701939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-13ea4e33-bb68-44ad-9235-64832725d0e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:21.701939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:21.701939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:21.701939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:21.701939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:21.701939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:21.701939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:21.701939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:21.701939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:21.701939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:21.701939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:21.701939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:21.701939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:21.701939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:21.701939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:21.701939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:21.702842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:21.702842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:21.702842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:21.702842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:21.702842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:21.702842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:21.702842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:21.702842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:21.702842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:21.702842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:21.702842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:21.702842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:21.702842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:21.702842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:21.702842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:21.702842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:21.703691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:21.703691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:21.703691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:21.703691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:21.703691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:21.703691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:21.703691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:21.703691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:21.703691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:21.703691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:21.703691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:21.703691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:21.706106 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 234/483] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:21.712711 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f181530-5ce7-4195-ada0-47d871d45dc3 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:21.786725 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:21.827971 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f181530-5ce7-4195-ada0-47d871d45dc3 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:21.863150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4153cb-b3d1-43ff-8c54-1b01934613de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:21.864198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4153cb-b3d1-43ff-8c54-1b01934613de None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:21.864718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4153cb-b3d1-43ff-8c54-1b01934613de None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:21.865205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4153cb-b3d1-43ff-8c54-1b01934613de None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:21.867060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4153cb-b3d1-43ff-8c54-1b01934613de None admin] RBAC: Adding query filter params (name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:21.867579 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4153cb-b3d1-43ff-8c54-1b01934613de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:21.869176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4153cb-b3d1-43ff-8c54-1b01934613de None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:21.878075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 250/484] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:23:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:21.888010 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea4e33-bb68-44ad-9235-64832725d0e2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:21.900445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8558d-7cc8-4f86-9f44-f2b95cab2663 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:21.903192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8558d-7cc8-4f86-9f44-f2b95cab2663 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:21.903790 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8558d-7cc8-4f86-9f44-f2b95cab2663 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:21.905393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 251/485] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:21.911027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0cd86-68fd-485d-89ab-f43cfbf1e39c None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:21.911577 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0cd86-68fd-485d-89ab-f43cfbf1e39c None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:21.912538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0cd86-68fd-485d-89ab-f43cfbf1e39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:21.932327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-fbb0cd86-68fd-485d-89ab-f43cfbf1e39c None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:21.934144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-fbb0cd86-68fd-485d-89ab-f43cfbf1e39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:21.974693 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:22.020431 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 252/486] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:22.023521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943f919-37bd-42f2-81ea-0ca916e68216 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:22.023862 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943f919-37bd-42f2-81ea-0ca916e68216 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:22.024153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943f919-37bd-42f2-81ea-0ca916e68216 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:22.029822 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 253/487] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:22.031688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f871e-8665-40b8-b966-d1937020e0f5 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:22.032050 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f871e-8665-40b8-b966-d1937020e0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:22.032338 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f871e-8665-40b8-b966-d1937020e0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:22.034154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea4e33-bb68-44ad-9235-64832725d0e2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:22.052565 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-023f871e-8665-40b8-b966-d1937020e0f5 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:22.056002 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-023f871e-8665-40b8-b966-d1937020e0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:22.085005 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62908d71-4c27-44f6-bfc6-2c7267c3b9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:22.085903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62908d71-4c27-44f6-bfc6-2c7267c3b9ac None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:22.086194 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62908d71-4c27-44f6-bfc6-2c7267c3b9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:22.086472 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62908d71-4c27-44f6-bfc6-2c7267c3b9ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:22.097670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62908d71-4c27-44f6-bfc6-2c7267c3b9ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:22.099154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62908d71-4c27-44f6-bfc6-2c7267c3b9ac None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:22.111984 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:22.120807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-62908d71-4c27-44f6-bfc6-2c7267c3b9ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:22.120807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:22.120807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:22.120807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:22.120807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:22.120807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:22.120807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:22.120807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:22.120807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:22.120807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:22.120807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:22.120807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:22.120807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:22.120807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:22.120807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:22.121650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:22.121650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:22.121650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:22.121650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:22.121650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:22.121650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:22.121650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:22.121650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:22.121650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:22.121650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:22.121650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:22.121650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:22.121650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:22.121650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:22.121650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:22.121650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:22.121650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:22.122534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:22.122534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:22.122534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:22.122534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:22.129021 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 235/488] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:22.129021 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62908d71-4c27-44f6-bfc6-2c7267c3b9ac None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:22.143050 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 254/489] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:23:22.144057 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f871e-8665-40b8-b966-d1937020e0f5 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:22.229238 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:22.235799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62908d71-4c27-44f6-bfc6-2c7267c3b9ac None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:22.273304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d34bb1-4f09-4e89-b1ea-e6e9d25d7563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:22.274285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d34bb1-4f09-4e89-b1ea-e6e9d25d7563 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:22.274709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d34bb1-4f09-4e89-b1ea-e6e9d25d7563 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:22.275087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d34bb1-4f09-4e89-b1ea-e6e9d25d7563 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:22.277417 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f871e-8665-40b8-b966-d1937020e0f5 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:22.285694 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d34bb1-4f09-4e89-b1ea-e6e9d25d7563 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:22.294532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-69d34bb1-4f09-4e89-b1ea-e6e9d25d7563 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:22.294532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:22.294532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:22.294532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:22.294532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:22.294532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:22.294532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:22.294532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:22.294532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:22.294532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:22.294532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:22.294532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:22.294532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:22.294532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:22.294532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:22.294532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:22.296521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:22.296521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:22.296521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:22.296521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:22.296521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:22.296521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:22.296521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:22.296521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:22.296521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:22.296521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:22.296521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:22.296521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:22.296521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:22.296521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:22.296521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:22.296521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:22.298368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:22.298368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:22.298368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:22.298368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:22.298368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:22.298368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:22.298368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:22.298368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:22.298368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:22.298368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:22.298368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:22.298368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:22.298368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 236/490] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:22.299656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d34bb1-4f09-4e89-b1ea-e6e9d25d7563 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:22.314311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212a041-a687-45d1-82c5-acaec0407628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:22.315021 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212a041-a687-45d1-82c5-acaec0407628 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:22.315314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212a041-a687-45d1-82c5-acaec0407628 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:22.315601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212a041-a687-45d1-82c5-acaec0407628 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:22.316448 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212a041-a687-45d1-82c5-acaec0407628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:22.316767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212a041-a687-45d1-82c5-acaec0407628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:22.318270 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212a041-a687-45d1-82c5-acaec0407628 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:22.331828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 255/491] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:22.332794 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212a041-a687-45d1-82c5-acaec0407628 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:22.384361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:22.420982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d34bb1-4f09-4e89-b1ea-e6e9d25d7563 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:22.445985 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212a041-a687-45d1-82c5-acaec0407628 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:22.458138 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05359f-a0e8-4e3f-beb8-b8543fca28c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:22.459166 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05359f-a0e8-4e3f-beb8-b8543fca28c0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:22.459873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05359f-a0e8-4e3f-beb8-b8543fca28c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:22.460554 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05359f-a0e8-4e3f-beb8-b8543fca28c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:22.471912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05359f-a0e8-4e3f-beb8-b8543fca28c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:22.481094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5f05359f-a0e8-4e3f-beb8-b8543fca28c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:22.481094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:22.481094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:22.481094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:22.481094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:22.481094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:22.481094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:22.481094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:22.481094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:22.481094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:22.481094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:22.481094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:22.481094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:22.481094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:22.481094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:22.481094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:22.482171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:22.482171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:22.482171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:22.482171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:22.482171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:22.482171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:22.482171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:22.482171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:22.482171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:22.482171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:22.482171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:22.482171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:22.482171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:22.482171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:22.482171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:22.482171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:22.483178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:22.483178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:22.483178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:22.483178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:22.483178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:22.483178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:22.483178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:22.483178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:22.483178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:22.483178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:22.483178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:22.483178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:22.484750 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f3673-9815-428f-b430-db8e2be27234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:22.485503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f3673-9815-428f-b430-db8e2be27234 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:22.485822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 237/492] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:22.486166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f3673-9815-428f-b430-db8e2be27234 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:22.486889 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05359f-a0e8-4e3f-beb8-b8543fca28c0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:22.487357 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f3673-9815-428f-b430-db8e2be27234 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:22.488302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f3673-9815-428f-b430-db8e2be27234 None admin] RBAC: Adding query filter params (name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:22.493036 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f3673-9815-428f-b430-db8e2be27234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:22.494542 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f3673-9815-428f-b430-db8e2be27234 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:22.504095 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 256/493] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:23:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:22.510021 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f3673-9815-428f-b430-db8e2be27234 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:22.513019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:22.607359 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05359f-a0e8-4e3f-beb8-b8543fca28c0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:22.623218 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f3673-9815-428f-b430-db8e2be27234 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:22.633207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:22.642443 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1836a-afb1-4c42-9ac1-c361f6a34cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:22.643271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1836a-afb1-4c42-9ac1-c361f6a34cfa None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:22.643556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1836a-afb1-4c42-9ac1-c361f6a34cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:22.643849 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1836a-afb1-4c42-9ac1-c361f6a34cfa None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:22.652691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1836a-afb1-4c42-9ac1-c361f6a34cfa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:22.654099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1836a-afb1-4c42-9ac1-c361f6a34cfa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:22.664008 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc4233-fe8a-49f5-bf5a-280a8c52dab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:22.664689 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc4233-fe8a-49f5-bf5a-280a8c52dab9 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:22.664978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc4233-fe8a-49f5-bf5a-280a8c52dab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:22.665261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc4233-fe8a-49f5-bf5a-280a8c52dab9 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:22.666017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcc4233-fe8a-49f5-bf5a-280a8c52dab9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:22.666331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcc4233-fe8a-49f5-bf5a-280a8c52dab9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:22.667724 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcc4233-fe8a-49f5-bf5a-280a8c52dab9 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:22.670157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5da1836a-afb1-4c42-9ac1-c361f6a34cfa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:23:22.670157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:22.670157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:22.670157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:22.670157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:22.670157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:22.670157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:22.670157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:22.670157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:22.670157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:22.670157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:22.670157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:22.670157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:22.670157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:22.670157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:22.670157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:22.671039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:22.671039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:22.671039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:22.671039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:22.671039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:22.671039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:22.671039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:22.671039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:22.671039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:22.671039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:22.671039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:22.671039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:22.671039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:22.671039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:22.671039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:22.671039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:22.671039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:22.671039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:22.672024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:22.672024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:22.672024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:22.672024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:22.672024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:22.672024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:22.672024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:22.672024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:22.672024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:23:22.672024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:22.676000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 238/494] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Jun 23 22:23:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:22.681650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 257/495] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:23:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:22.681775 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1836a-afb1-4c42-9ac1-c361f6a34cfa None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:22.681850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646486c-a618-4cba-a4a8-04328c46a8c3 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:22.681850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646486c-a618-4cba-a4a8-04328c46a8c3 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:22.681850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646486c-a618-4cba-a4a8-04328c46a8c3 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:22.681850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 258/496] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:22.681850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646486c-a618-4cba-a4a8-04328c46a8c3 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:22.766840 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:22.786257 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646486c-a618-4cba-a4a8-04328c46a8c3 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:22.794824 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1836a-afb1-4c42-9ac1-c361f6a34cfa None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:22.819204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f055d-4a61-43ba-9113-a6e99aa179ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:22.820165 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f055d-4a61-43ba-9113-a6e99aa179ee None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:22.820846 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f055d-4a61-43ba-9113-a6e99aa179ee None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:22.821254 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f055d-4a61-43ba-9113-a6e99aa179ee None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:22.822338 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f055d-4a61-43ba-9113-a6e99aa179ee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:22.822671 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f055d-4a61-43ba-9113-a6e99aa179ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:22.824398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f055d-4a61-43ba-9113-a6e99aa179ee None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:22.831600 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bfbc7-3b8f-4bc8-9481-ad9907a99ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:22.832500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bfbc7-3b8f-4bc8-9481-ad9907a99ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:22.832852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bfbc7-3b8f-4bc8-9481-ad9907a99ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:22.833199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bfbc7-3b8f-4bc8-9481-ad9907a99ba3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:22.844951 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bfbc7-3b8f-4bc8-9481-ad9907a99ba3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:22.846486 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bfbc7-3b8f-4bc8-9481-ad9907a99ba3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:22.866031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a78bfbc7-3b8f-4bc8-9481-ad9907a99ba3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:22.866031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:22.866031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:22.866031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:22.866031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:22.866031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:22.866031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:22.866031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:22.866031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:22.866031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:22.866031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:22.866031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:22.866031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:22.866031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:22.866031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:22.867675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:22.867675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:22.867675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:22.867675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:22.867675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:22.867675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:22.867675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:22.867675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:22.867675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:22.867675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:22.867675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:22.867675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:22.867675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:22.867675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:22.867675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:22.867675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:22.867675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:22.868531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:22.868531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:22.868531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:22.868531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:22.868531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 239/497] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:22.868531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bfbc7-3b8f-4bc8-9481-ad9907a99ba3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:22.869416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 259/498] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Jun 23 22:23:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:22.870860 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dd094-8c2d-4b7a-bb75-259a26007fb7 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:22.871336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dd094-8c2d-4b7a-bb75-259a26007fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:22.871997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5dd094-8c2d-4b7a-bb75-259a26007fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:22.891284 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-4f5dd094-8c2d-4b7a-bb75-259a26007fb7 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:22.892469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-4f5dd094-8c2d-4b7a-bb75-259a26007fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:22.916825 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:22.961967 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 260/499] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:22.962796 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5dd094-8c2d-4b7a-bb75-259a26007fb7 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:22.974382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bfbc7-3b8f-4bc8-9481-ad9907a99ba3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:23.000837 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:23.009616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfc0c8-a1b0-4db5-a61d-557fd5cfb941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:23.010401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfc0c8-a1b0-4db5-a61d-557fd5cfb941 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:23.010683 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfc0c8-a1b0-4db5-a61d-557fd5cfb941 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:23.010987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfc0c8-a1b0-4db5-a61d-557fd5cfb941 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:23.021597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bfc0c8-a1b0-4db5-a61d-557fd5cfb941 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:23.023460 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bfc0c8-a1b0-4db5-a61d-557fd5cfb941 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:23.040691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d6bfc0c8-a1b0-4db5-a61d-557fd5cfb941 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:23.040691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:23.040691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:23.040691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:23.040691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:23.040691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:23.040691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:23.040691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:23.040691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:23.040691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:23.040691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:23.040691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:23.040691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:23.040691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:23.040691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:23.041701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:23.041701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:23.041701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:23.041701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:23.041701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:23.041701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:23.041701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:23.041701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:23.041701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:23.041701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:23.041701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:23.041701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:23.041701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:23.041701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:23.041701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:23.041701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:23.041701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:23.042727 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:23.042727 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:23.042727 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:23.042727 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:23.042727 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 240/500] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:23.042727 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfc0c8-a1b0-4db5-a61d-557fd5cfb941 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:23.072976 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5dd094-8c2d-4b7a-bb75-259a26007fb7 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:23.106445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367d3b8-1c5a-4a66-839a-db7e3a42aa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:23.107142 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367d3b8-1c5a-4a66-839a-db7e3a42aa5c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:23.107429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367d3b8-1c5a-4a66-839a-db7e3a42aa5c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:23.107713 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367d3b8-1c5a-4a66-839a-db7e3a42aa5c None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:23.108434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367d3b8-1c5a-4a66-839a-db7e3a42aa5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:23.108747 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367d3b8-1c5a-4a66-839a-db7e3a42aa5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:23.110135 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367d3b8-1c5a-4a66-839a-db7e3a42aa5c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:23.121396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 261/501] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:23.122673 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4244bee-af05-42dd-a558-e9e66b785980 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:23.122987 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4244bee-af05-42dd-a558-e9e66b785980 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:23.123279 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4244bee-af05-42dd-a558-e9e66b785980 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:23.124974 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 262/502] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:23.125643 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4244bee-af05-42dd-a558-e9e66b785980 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:23.145858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfc0c8-a1b0-4db5-a61d-557fd5cfb941 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:23.148787 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:23.181804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd017476-10e8-443c-a146-21ef32479daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:23.182452 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd017476-10e8-443c-a146-21ef32479daa None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:23.182738 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd017476-10e8-443c-a146-21ef32479daa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:23.183053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd017476-10e8-443c-a146-21ef32479daa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:23.234142 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4244bee-af05-42dd-a558-e9e66b785980 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:23.234879 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:23.255790 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd017476-10e8-443c-a146-21ef32479daa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:23.257578 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd017476-10e8-443c-a146-21ef32479daa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:23.268527 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc586b-7206-4980-89ea-285cd1e132c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:23.269327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc586b-7206-4980-89ea-285cd1e132c6 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:23.269616 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc586b-7206-4980-89ea-285cd1e132c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:23.269898 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc586b-7206-4980-89ea-285cd1e132c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:23.280858 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc586b-7206-4980-89ea-285cd1e132c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:23.288983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-7abc586b-7206-4980-89ea-285cd1e132c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:23.288983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:23.288983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:23.288983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:23.288983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:23.288983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:23.288983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:23.288983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:23.288983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:23.288983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:23.288983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:23.288983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:23.288983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:23.288983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:23.288983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:23.288983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:23.289845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:23.289845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:23.289845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:23.289845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:23.289845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:23.289845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:23.289845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:23.289845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:23.289845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:23.289845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:23.289845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:23.289845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:23.289845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:23.289845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:23.289845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:23.289845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:23.295255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:23.295255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:23.295255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:23.295255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:23.295255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:23.295255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:23.295255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:23.295255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:23.295255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:23.295255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:23.295255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:23.295255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:23.295255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 263/503] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:23.295255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc586b-7206-4980-89ea-285cd1e132c6 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:23.354949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 241/504] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:23.356001 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd017476-10e8-443c-a146-21ef32479daa None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:23.373032 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:23.394450 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc586b-7206-4980-89ea-285cd1e132c6 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:23.428794 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae395c-afb9-4100-9c43-13b261c88921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:23.429637 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae395c-afb9-4100-9c43-13b261c88921 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:23.430185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae395c-afb9-4100-9c43-13b261c88921 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:23.430185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae395c-afb9-4100-9c43-13b261c88921 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:23.430909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae395c-afb9-4100-9c43-13b261c88921 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:23.431253 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae395c-afb9-4100-9c43-13b261c88921 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:23.432646 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae395c-afb9-4100-9c43-13b261c88921 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:23.433967 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:23.445006 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 264/505] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:23.446225 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f029fa0-9fd7-4ef3-8ad9-874f28e9f756 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:23.446518 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f029fa0-9fd7-4ef3-8ad9-874f28e9f756 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:23.446800 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f029fa0-9fd7-4ef3-8ad9-874f28e9f756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:23.457836 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd017476-10e8-443c-a146-21ef32479daa None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:23.462322 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-9f029fa0-9fd7-4ef3-8ad9-874f28e9f756 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:23.463549 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-9f029fa0-9fd7-4ef3-8ad9-874f28e9f756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:23.492324 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed638439-f2bd-4d02-a4b0-f2e2e5ecf810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:23.493118 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed638439-f2bd-4d02-a4b0-f2e2e5ecf810 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:23.493415 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed638439-f2bd-4d02-a4b0-f2e2e5ecf810 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:23.493693 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed638439-f2bd-4d02-a4b0-f2e2e5ecf810 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:23.500626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed638439-f2bd-4d02-a4b0-f2e2e5ecf810 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:23.502079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed638439-f2bd-4d02-a4b0-f2e2e5ecf810 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:23.517334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ed638439-f2bd-4d02-a4b0-f2e2e5ecf810 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 22:23:23.517334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:23.517334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:23.517334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:23.517334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:23.517334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:23.517334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:23.517334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:23.517334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:23.517334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:23.517334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:23.517334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:23.517334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:23.517334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:23.517334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:23.517334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:23.518225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:23.518225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:23.518225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:23.518225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:23.518225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:23.518225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:23.518225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:23.518225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:23.518225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:23.518225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:23.518225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:23.518225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:23.518225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:23.518225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:23.518225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:23.518225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:23.518225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:23.518225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:23.519157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:23.519157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:23.519157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:23.519157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:23.519157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:23.519157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:23.519157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:23.519157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:23.519157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 22:23:23.519157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:23.519157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 242/506] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:23.519701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed638439-f2bd-4d02-a4b0-f2e2e5ecf810 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:23.533612 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 265/507] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:23.534372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f029fa0-9fd7-4ef3-8ad9-874f28e9f756 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:23.600938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:23.626547 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed638439-f2bd-4d02-a4b0-f2e2e5ecf810 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:23.634224 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f029fa0-9fd7-4ef3-8ad9-874f28e9f756 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:23.661901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677046b-72aa-416a-9c47-e8200f783ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:23.663198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677046b-72aa-416a-9c47-e8200f783ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:23.663561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677046b-72aa-416a-9c47-e8200f783ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:23.663856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677046b-72aa-416a-9c47-e8200f783ae0 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:23.664605 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677046b-72aa-416a-9c47-e8200f783ae0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:23.664921 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677046b-72aa-416a-9c47-e8200f783ae0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:23.666412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677046b-72aa-416a-9c47-e8200f783ae0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:23.668937 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d22f3a-ff16-4f5c-afe9-10f0d8a4410f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:23.669730 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d22f3a-ff16-4f5c-afe9-10f0d8a4410f None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:23.670015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d22f3a-ff16-4f5c-afe9-10f0d8a4410f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:23.670294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d22f3a-ff16-4f5c-afe9-10f0d8a4410f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:23.674195 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 243/508] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:23:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:23.675072 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677046b-72aa-416a-9c47-e8200f783ae0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:23.677542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:23.681704 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d22f3a-ff16-4f5c-afe9-10f0d8a4410f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:23.683184 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d22f3a-ff16-4f5c-afe9-10f0d8a4410f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:23.701087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-65d22f3a-ff16-4f5c-afe9-10f0d8a4410f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:23.701087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:23.701087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:23.701087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:23.701087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:23.701087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:23.701087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:23.701087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:23.701087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:23.701087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:23.701087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:23.701087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:23.701087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:23.701087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:23.701087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:23.702128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:23.702128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:23.702128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:23.702128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:23.702128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:23.702128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:23.702128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:23.702128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:23.702128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:23.702128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:23.702128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:23.702128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:23.702128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:23.702128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:23.702128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:23.702128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:23.702128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:23.703071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:23.703071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:23.703071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:23.703071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:23.703071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 266/509] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:23.703341 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d22f3a-ff16-4f5c-afe9-10f0d8a4410f None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:23.776681 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677046b-72aa-416a-9c47-e8200f783ae0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:23.803165 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d22f3a-ff16-4f5c-afe9-10f0d8a4410f None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:23.809019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633c91b-25f0-41cf-a0e4-4043d8ac5c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:23.809804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633c91b-25f0-41cf-a0e4-4043d8ac5c5e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:23.810087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633c91b-25f0-41cf-a0e4-4043d8ac5c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:23.810362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633c91b-25f0-41cf-a0e4-4043d8ac5c5e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:23.817686 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a633c91b-25f0-41cf-a0e4-4043d8ac5c5e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:23.819697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a633c91b-25f0-41cf-a0e4-4043d8ac5c5e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:23.823549 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:23.834050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a633c91b-25f0-41cf-a0e4-4043d8ac5c5e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 22:23:23.834050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:23.834050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:23.834050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:23.834050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:23.834050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:23.834050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:23.834050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:23.834050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:23.834050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:23.834050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:23.834050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:23.834050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:23.834050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:23.834050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:23.834050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:23.835613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:23.835613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:23.835613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:23.835613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:23.835613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:23.835613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:23.835613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:23.835613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:23.835613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:23.835613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:23.835613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:23.835613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:23.835613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:23.835613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:23.835613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:23.835613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:23.835613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:23.835613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:23.836492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:23.836492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:23.836492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:23.836492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:23.836492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:23.836492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:23.836492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:23.836492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:23.836492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 22:23:23.836492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:23.836492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 244/510] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:23.837127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633c91b-25f0-41cf-a0e4-4043d8ac5c5e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:23.844525 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a8e87-2a1d-4661-acce-054a2df1d701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:23.845200 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a8e87-2a1d-4661-acce-054a2df1d701 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:23.845491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a8e87-2a1d-4661-acce-054a2df1d701 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:23.845774 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a8e87-2a1d-4661-acce-054a2df1d701 None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:23.846617 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a8e87-2a1d-4661-acce-054a2df1d701 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:23.846942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a8e87-2a1d-4661-acce-054a2df1d701 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:23.848437 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344a8e87-2a1d-4661-acce-054a2df1d701 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:23.873780 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 267/511] 10.222.0.43 () {56 vars in 1137 bytes} [Thu Jun 23 22:23:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:23.873780 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a8e87-2a1d-4661-acce-054a2df1d701 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:23.899667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:23.938285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633c91b-25f0-41cf-a0e4-4043d8ac5c5e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:23.969898 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a8e87-2a1d-4661-acce-054a2df1d701 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:23.971528 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dea4d6-6b57-420d-8f4f-c27e91569db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:23.972320 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dea4d6-6b57-420d-8f4f-c27e91569db4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:23.972609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dea4d6-6b57-420d-8f4f-c27e91569db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:23.972892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dea4d6-6b57-420d-8f4f-c27e91569db4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:23.983267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dea4d6-6b57-420d-8f4f-c27e91569db4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:23.984726 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dea4d6-6b57-420d-8f4f-c27e91569db4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:24.001299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a3dea4d6-6b57-420d-8f4f-c27e91569db4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:24.001299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:24.001299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:24.001299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:24.001299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:24.001299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:24.001299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:24.001299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:24.001299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:24.001299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:24.001299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:24.001299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:24.001299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:24.001299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:24.001299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:24.002175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:24.002175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:24.002175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:24.002175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:24.002175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:24.002175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:24.002175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:24.002175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:24.002175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:24.002175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:24.002175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:24.002175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:24.002175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:24.002175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:24.002175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:24.002175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:24.002175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:24.003069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:24.003069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:24.003069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:24.003069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:24.003069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 245/512] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:24.003333 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dea4d6-6b57-420d-8f4f-c27e91569db4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:24.004155 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2a3f7-6235-496f-a0f3-d2103dba34d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:24.004957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2a3f7-6235-496f-a0f3-d2103dba34d8 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:24.005244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2a3f7-6235-496f-a0f3-d2103dba34d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:24.005582 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2a3f7-6235-496f-a0f3-d2103dba34d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:24.016886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2a3f7-6235-496f-a0f3-d2103dba34d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:24.018379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2a3f7-6235-496f-a0f3-d2103dba34d8 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:24.035977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-31d2a3f7-6235-496f-a0f3-d2103dba34d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:24.035977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:24.035977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:24.035977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:24.035977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:24.035977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:24.035977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:24.035977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:24.035977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:24.035977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:24.035977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:24.035977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:24.035977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:24.035977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:24.035977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:24.036938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:24.036938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:24.036938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:24.036938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:24.036938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:24.036938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:24.036938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:24.036938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:24.036938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:24.036938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:24.036938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:24.036938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:24.036938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:24.036938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:24.036938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:24.036938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:24.036938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:24.038066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:24.038066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:24.038066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:24.038066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:24.038066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 268/513] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:24.038066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2a3f7-6235-496f-a0f3-d2103dba34d8 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:24.068291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:24.103273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dea4d6-6b57-420d-8f4f-c27e91569db4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:24.117121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:24.137974 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2a3f7-6235-496f-a0f3-d2103dba34d8 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:24.138789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d84c6-7f01-4036-8e50-9f13ce68b517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:24.139500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d84c6-7f01-4036-8e50-9f13ce68b517 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:24.139790 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d84c6-7f01-4036-8e50-9f13ce68b517 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:24.140769 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d84c6-7f01-4036-8e50-9f13ce68b517 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:24.140769 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d84c6-7f01-4036-8e50-9f13ce68b517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:24.141098 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d84c6-7f01-4036-8e50-9f13ce68b517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:24.142560 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d84c6-7f01-4036-8e50-9f13ce68b517 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:24.155020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 246/514] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:24.155867 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d84c6-7f01-4036-8e50-9f13ce68b517 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:24.171571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bda0d1-dab3-4793-8ad4-e9d9a1963e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:24.172242 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bda0d1-dab3-4793-8ad4-e9d9a1963e4b None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:24.172525 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bda0d1-dab3-4793-8ad4-e9d9a1963e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:24.172803 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bda0d1-dab3-4793-8ad4-e9d9a1963e4b None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:24.173641 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bda0d1-dab3-4793-8ad4-e9d9a1963e4b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:24.173951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bda0d1-dab3-4793-8ad4-e9d9a1963e4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:24.175407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bda0d1-dab3-4793-8ad4-e9d9a1963e4b None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:24.192700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 269/515] 10.222.0.43 () {56 vars in 1137 bytes} [Thu Jun 23 22:23:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:24.193484 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bda0d1-dab3-4793-8ad4-e9d9a1963e4b None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:24.253335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d84c6-7f01-4036-8e50-9f13ce68b517 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:24.279788 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:24.289432 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611862b4-3264-4ee5-b904-d066e3789107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:24.290100 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611862b4-3264-4ee5-b904-d066e3789107 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:24.290393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611862b4-3264-4ee5-b904-d066e3789107 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:24.290680 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611862b4-3264-4ee5-b904-d066e3789107 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:24.294738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bda0d1-dab3-4793-8ad4-e9d9a1963e4b None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:24.330048 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96238f-614a-45e2-9e14-b68f789651f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:24.330652 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96238f-614a-45e2-9e14-b68f789651f6 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:24.330964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96238f-614a-45e2-9e14-b68f789651f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:24.331255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96238f-614a-45e2-9e14-b68f789651f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:24.331981 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96238f-614a-45e2-9e14-b68f789651f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:24.332297 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96238f-614a-45e2-9e14-b68f789651f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:24.333792 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96238f-614a-45e2-9e14-b68f789651f6 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:24.336647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:24.345207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 270/516] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:24.346011 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96238f-614a-45e2-9e14-b68f789651f6 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:24.364940 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611862b4-3264-4ee5-b904-d066e3789107 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:24.366978 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611862b4-3264-4ee5-b904-d066e3789107 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:24.443110 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96238f-614a-45e2-9e14-b68f789651f6 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:24.463587 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 247/517] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:24.464423 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611862b4-3264-4ee5-b904-d066e3789107 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:24.479855 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2eee9-1067-4378-923d-1c5a5ea99378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:24.480524 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2eee9-1067-4378-923d-1c5a5ea99378 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:24.480810 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2eee9-1067-4378-923d-1c5a5ea99378 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:24.481086 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2eee9-1067-4378-923d-1c5a5ea99378 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:24.481808 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2eee9-1067-4378-923d-1c5a5ea99378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:24.482116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2eee9-1067-4378-923d-1c5a5ea99378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:24.483545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2eee9-1067-4378-923d-1c5a5ea99378 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:24.495903 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 271/518] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:24.496518 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2eee9-1067-4378-923d-1c5a5ea99378 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:24.508267 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:24.542539 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:24.567452 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611862b4-3264-4ee5-b904-d066e3789107 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:24.595086 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2eee9-1067-4378-923d-1c5a5ea99378 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:24.600577 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b43362-cf24-41cf-a3e6-ba86cf1148fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:24.601362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b43362-cf24-41cf-a3e6-ba86cf1148fc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:24.601647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b43362-cf24-41cf-a3e6-ba86cf1148fc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:24.601926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b43362-cf24-41cf-a3e6-ba86cf1148fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:24.608333 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b43362-cf24-41cf-a3e6-ba86cf1148fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:24.609752 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b43362-cf24-41cf-a3e6-ba86cf1148fc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:24.624120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b6b43362-cf24-41cf-a3e6-ba86cf1148fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:24.624120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:24.624120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:24.624120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:24.624120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:24.624120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:24.624120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:24.624120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:24.624120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:24.624120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:24.624120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:24.624120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:24.624120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:24.624120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:24.624120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:24.624993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:24.624993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:24.624993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:24.624993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:24.624993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:24.624993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:24.624993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:24.624993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:24.624993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:24.624993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:24.624993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:24.624993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:24.624993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:24.624993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:24.624993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:24.624993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:24.624993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:24.625861 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:24.625861 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:24.625861 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:24.625861 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:24.625861 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:24.625861 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:24.625861 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:24.625861 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:24.627072 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 248/519] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:24.628049 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b43362-cf24-41cf-a3e6-ba86cf1148fc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:24.629328 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188bbd5-498e-433f-ab56-8f70612fea4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:24.630088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188bbd5-498e-433f-ab56-8f70612fea4d None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:24.630382 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188bbd5-498e-433f-ab56-8f70612fea4d None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:24.630667 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188bbd5-498e-433f-ab56-8f70612fea4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:24.701635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188bbd5-498e-433f-ab56-8f70612fea4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:24.704224 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188bbd5-498e-433f-ab56-8f70612fea4d None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:24.715086 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:24.732717 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b43362-cf24-41cf-a3e6-ba86cf1148fc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:24.766994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f2961a-2607-415a-8d93-89caff017161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:24.767662 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2961a-2607-415a-8d93-89caff017161 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:24.767958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2961a-2607-415a-8d93-89caff017161 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:24.768240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2961a-2607-415a-8d93-89caff017161 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:24.768941 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f2961a-2607-415a-8d93-89caff017161 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:24.770375 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f2961a-2607-415a-8d93-89caff017161 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:24.777834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:24.791239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 249/520] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 22:23:24 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:23:24.792053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f2961a-2607-415a-8d93-89caff017161 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:24.809634 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 272/521] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:24.809634 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188bbd5-498e-433f-ab56-8f70612fea4d None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:24.913951 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f2961a-2607-415a-8d93-89caff017161 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:24.919221 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188bbd5-498e-433f-ab56-8f70612fea4d None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:24.930705 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:24.957262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80434050-cf0f-43fd-a157-b25a5be3d9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:24.957465 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449aad9-5145-4242-be13-4e3b1940e707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:24.957919 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80434050-cf0f-43fd-a157-b25a5be3d9da None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:24.958202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80434050-cf0f-43fd-a157-b25a5be3d9da None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:24.958295 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449aad9-5145-4242-be13-4e3b1940e707 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:24.958480 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80434050-cf0f-43fd-a157-b25a5be3d9da None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:24.958580 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449aad9-5145-4242-be13-4e3b1940e707 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:24.958772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449aad9-5145-4242-be13-4e3b1940e707 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:24.968278 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80434050-cf0f-43fd-a157-b25a5be3d9da None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:24.969571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80434050-cf0f-43fd-a157-b25a5be3d9da None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:24.971300 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449aad9-5145-4242-be13-4e3b1940e707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:24.972735 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449aad9-5145-4242-be13-4e3b1940e707 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:24.989037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-80434050-cf0f-43fd-a157-b25a5be3d9da None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 22:23:24.989037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:24.989037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:24.989037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:24.989037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:24.989037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:24.989037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:24.989037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:24.989037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:24.989037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:24.989037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:24.989037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:24.989037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:24.989037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:24.989037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:24.989037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:24.989959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:24.989959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:24.989959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:24.989959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:24.989959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:24.989959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:24.989959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:24.989959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:24.989959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:24.989959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:24.989959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:24.989959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:24.989959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:24.989959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:24.989959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:24.989959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:24.989959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:24.989959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:24.991424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:24.991424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:24.991424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:24.991424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:24.991424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:24.991424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:24.991424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:24.991424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:24.991424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 22:23:24.991424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:24.992393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3449aad9-5145-4242-be13-4e3b1940e707 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:24.992393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:24.992393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:24.992393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:24.992393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:24.992393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:24.992393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:24.992393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:24.992393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:24.992393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:24.992393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:24.992393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:24.992393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:24.992393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:24.992393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:24.993150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:24.993150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:24.993150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:24.993150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:24.993150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:24.993150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:24.993150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:24.993150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:24.993150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:24.993150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:24.993150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:24.993150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:24.993150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:24.993150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:24.993150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:24.993150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:24.993150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:24.993985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:24.993985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:24.993985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:24.993985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:24.994187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 273/522] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:24.994377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 250/523] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:24.994462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80434050-cf0f-43fd-a157-b25a5be3d9da None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:24.994531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449aad9-5145-4242-be13-4e3b1940e707 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:25.045975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:25.100291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80434050-cf0f-43fd-a157-b25a5be3d9da None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:25.110827 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449aad9-5145-4242-be13-4e3b1940e707 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:25.137761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f61c34-bc93-4fae-96c6-da97a0d0de6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:25.138439 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f61c34-bc93-4fae-96c6-da97a0d0de6c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:25.138730 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f61c34-bc93-4fae-96c6-da97a0d0de6c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:25.139046 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f61c34-bc93-4fae-96c6-da97a0d0de6c None admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:25.139853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f61c34-bc93-4fae-96c6-da97a0d0de6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:25.140173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f61c34-bc93-4fae-96c6-da97a0d0de6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:25.141742 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f61c34-bc93-4fae-96c6-da97a0d0de6c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:25.151161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939c4db-4c12-435c-98c1-68a33e8dd657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:25.151962 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939c4db-4c12-435c-98c1-68a33e8dd657 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:25.152249 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939c4db-4c12-435c-98c1-68a33e8dd657 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:25.152530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939c4db-4c12-435c-98c1-68a33e8dd657 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:25.165948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 274/524] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:25.165948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f61c34-bc93-4fae-96c6-da97a0d0de6c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:25.169063 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939c4db-4c12-435c-98c1-68a33e8dd657 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:25.170529 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939c4db-4c12-435c-98c1-68a33e8dd657 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:25.188322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-8939c4db-4c12-435c-98c1-68a33e8dd657 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:25.188322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:25.188322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:25.188322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:25.188322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:25.188322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:25.188322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:25.188322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:25.188322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:25.188322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:25.188322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:25.188322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:25.188322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:25.188322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:25.188322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:25.189401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:25.189401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:25.189401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:25.189401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:25.189401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:25.189401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:25.189401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:25.189401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:25.189401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:25.189401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:25.189401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:25.189401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:25.189401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:25.189401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:25.189401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:25.189401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:25.189401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:25.190384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:25.190384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:25.190384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:25.190384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:25.190384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 251/525] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:25.191802 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939c4db-4c12-435c-98c1-68a33e8dd657 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:25.197380 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:25.281067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f61c34-bc93-4fae-96c6-da97a0d0de6c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:25.294632 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:25.305129 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939c4db-4c12-435c-98c1-68a33e8dd657 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:25.324170 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff844ae-b57f-449c-b5e5-8753a9b3a1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:25.324858 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff844ae-b57f-449c-b5e5-8753a9b3a1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:25.325149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff844ae-b57f-449c-b5e5-8753a9b3a1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:25.325433 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff844ae-b57f-449c-b5e5-8753a9b3a1c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:25.326585 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff844ae-b57f-449c-b5e5-8753a9b3a1c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:25.326910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff844ae-b57f-449c-b5e5-8753a9b3a1c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:25.328763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff844ae-b57f-449c-b5e5-8753a9b3a1c6 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:25.340792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6c5f1-55c1-4fb0-ae15-4edcb6eb1416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:25.341471 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6c5f1-55c1-4fb0-ae15-4edcb6eb1416 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:25.341739 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6c5f1-55c1-4fb0-ae15-4edcb6eb1416 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:25.342018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6c5f1-55c1-4fb0-ae15-4edcb6eb1416 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:25.342859 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6c5f1-55c1-4fb0-ae15-4edcb6eb1416 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:25.343206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6c5f1-55c1-4fb0-ae15-4edcb6eb1416 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:25.343314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 275/526] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:25.347965 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff844ae-b57f-449c-b5e5-8753a9b3a1c6 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:25.348081 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6c5f1-55c1-4fb0-ae15-4edcb6eb1416 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:25.388524 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 252/527] 10.222.0.43 () {56 vars in 1139 bytes} [Thu Jun 23 22:23:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:25.388524 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6c5f1-55c1-4fb0-ae15-4edcb6eb1416 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:25.457961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:25.472102 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff844ae-b57f-449c-b5e5-8753a9b3a1c6 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:25.492416 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6c5f1-55c1-4fb0-ae15-4edcb6eb1416 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:25.508608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdf375-7807-417e-bb01-043863d4d8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:25.509283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdf375-7807-417e-bb01-043863d4d8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:25.509575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdf375-7807-417e-bb01-043863d4d8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:25.509858 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdf375-7807-417e-bb01-043863d4d8c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:25.510692 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbdf375-7807-417e-bb01-043863d4d8c4 None admin] RBAC: Adding query filter params (role.id=2a71a120d6e34544aa5a2031ddf65ec8, scope.project.id=56da0975019c44409b53be473131143a, user.id=35d686923ef541819de3c77847d38511) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:25.511034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbdf375-7807-417e-bb01-043863d4d8c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:25.512467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbdf375-7807-417e-bb01-043863d4d8c4 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:25.526953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 276/528] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:23:25 2022] GET /identity/v3/role_assignments?user.id=35d686923ef541819de3c77847d38511&scope.project.id=56da0975019c44409b53be473131143a&role.id=2a71a120d6e34544aa5a2031ddf65ec8 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:25.526953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdf375-7807-417e-bb01-043863d4d8c4 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:25.538877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74a53a-f545-446b-9119-8aeaf99adafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:25.539691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74a53a-f545-446b-9119-8aeaf99adafe None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:25.540181 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74a53a-f545-446b-9119-8aeaf99adafe None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:25.540502 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74a53a-f545-446b-9119-8aeaf99adafe None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:25.541315 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74a53a-f545-446b-9119-8aeaf99adafe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:25.541700 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74a53a-f545-446b-9119-8aeaf99adafe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:25.543337 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74a53a-f545-446b-9119-8aeaf99adafe None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:25.561221 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 253/529] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:25.562287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74a53a-f545-446b-9119-8aeaf99adafe None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:25.579487 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:25.651326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdf375-7807-417e-bb01-043863d4d8c4 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:25.695971 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1764389a-b558-4c8b-91df-d58eccc7350e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:25.696931 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764389a-b558-4c8b-91df-d58eccc7350e None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:25.697215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764389a-b558-4c8b-91df-d58eccc7350e None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:25.697499 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764389a-b558-4c8b-91df-d58eccc7350e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:25.707561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1764389a-b558-4c8b-91df-d58eccc7350e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:25.709430 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1764389a-b558-4c8b-91df-d58eccc7350e None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:25.711251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74a53a-f545-446b-9119-8aeaf99adafe None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:25.725247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-1764389a-b558-4c8b-91df-d58eccc7350e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:25.725247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:25.725247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:25.725247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:25.725247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:25.725247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:25.725247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:25.725247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:25.725247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:25.725247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:25.725247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:25.725247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:25.725247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:25.725247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:25.725247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:25.726303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:25.726303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:25.726303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:25.726303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:25.726303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:25.726303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:25.726303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:25.726303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:25.726303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:25.726303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:25.726303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:25.726303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:25.726303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:25.726303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:25.726303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:25.726303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:25.726303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:25.727193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:25.727193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:25.727193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:25.727193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:25.727193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:25.727193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:25.727193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:25.727193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:25.729641 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 277/530] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:25.730622 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1764389a-b558-4c8b-91df-d58eccc7350e None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:25.748630 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:25.750639 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83bb54-0226-4175-9e5f-41971e462b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:25.751536 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83bb54-0226-4175-9e5f-41971e462b91 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:25.751668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83bb54-0226-4175-9e5f-41971e462b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:25.751869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83bb54-0226-4175-9e5f-41971e462b91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:25.824797 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83bb54-0226-4175-9e5f-41971e462b91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:25.826505 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83bb54-0226-4175-9e5f-41971e462b91 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:25.836585 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1764389a-b558-4c8b-91df-d58eccc7350e None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:25.842073 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:25.875708 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0918f-dde0-40ed-acb8-715df1a5a114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:25.876960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0918f-dde0-40ed-acb8-715df1a5a114 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:25.877441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0918f-dde0-40ed-acb8-715df1a5a114 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:25.877913 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0918f-dde0-40ed-acb8-715df1a5a114 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:25.885474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0918f-dde0-40ed-acb8-715df1a5a114 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:25.887303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0918f-dde0-40ed-acb8-715df1a5a114 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:25.902909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-1bd0918f-dde0-40ed-acb8-715df1a5a114 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:25.902909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:25.902909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:25.902909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:25.902909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:25.902909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:25.902909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:25.902909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:25.902909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:25.902909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:25.902909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:25.902909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:25.902909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:25.902909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:25.902909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:25.903811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:25.903811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:25.903811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:25.903811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:25.903811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:25.903811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:25.903811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:25.903811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:25.903811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:25.903811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:25.903811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:25.903811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:25.903811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:25.903811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:25.903811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:25.903811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:25.903811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:25.904656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:25.904656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:25.904656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:25.904656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:25.904656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:25.904656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:25.904656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:25.904656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:25.907160 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 278/531] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:25.908269 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0918f-dde0-40ed-acb8-715df1a5a114 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:25.945877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 254/532] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:25.946793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83bb54-0226-4175-9e5f-41971e462b91 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:25.986848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:26.020137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0918f-dde0-40ed-acb8-715df1a5a114 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:26.058315 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b6e2a-a5ed-4774-b2e7-e8a0fc2b359d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:26.059463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b6e2a-a5ed-4774-b2e7-e8a0fc2b359d None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:26.059894 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b6e2a-a5ed-4774-b2e7-e8a0fc2b359d None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:26.060355 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b6e2a-a5ed-4774-b2e7-e8a0fc2b359d None admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:26.061295 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b6e2a-a5ed-4774-b2e7-e8a0fc2b359d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:26.061752 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b6e2a-a5ed-4774-b2e7-e8a0fc2b359d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:26.063366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b6e2a-a5ed-4774-b2e7-e8a0fc2b359d None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:26.077882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 279/533] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:26.079039 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b6e2a-a5ed-4774-b2e7-e8a0fc2b359d None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:26.083325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83bb54-0226-4175-9e5f-41971e462b91 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:26.097350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:26.123625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831cc37-6ce4-4bcf-8826-f8b24b40ebca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:26.123625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831cc37-6ce4-4bcf-8826-f8b24b40ebca None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:26.123625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831cc37-6ce4-4bcf-8826-f8b24b40ebca None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:26.123625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831cc37-6ce4-4bcf-8826-f8b24b40ebca None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:26.128899 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831cc37-6ce4-4bcf-8826-f8b24b40ebca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:26.129389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831cc37-6ce4-4bcf-8826-f8b24b40ebca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:26.131741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831cc37-6ce4-4bcf-8826-f8b24b40ebca None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:26.155964 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 255/534] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:26.157007 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831cc37-6ce4-4bcf-8826-f8b24b40ebca None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:26.194886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b6e2a-a5ed-4774-b2e7-e8a0fc2b359d None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:26.226646 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:26.231808 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ac417-054b-4e96-a2be-90bbb5e8158a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:26.232616 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ac417-054b-4e96-a2be-90bbb5e8158a None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:26.233251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ac417-054b-4e96-a2be-90bbb5e8158a None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:26.233524 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ac417-054b-4e96-a2be-90bbb5e8158a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:26.246812 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ac417-054b-4e96-a2be-90bbb5e8158a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:26.248421 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ac417-054b-4e96-a2be-90bbb5e8158a None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:26.268799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-7f8ac417-054b-4e96-a2be-90bbb5e8158a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:26.268799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:26.268799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:26.268799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:26.268799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:26.268799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:26.268799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:26.268799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:26.268799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:26.268799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:26.268799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:26.268799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:26.268799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:26.268799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:26.268799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:26.269854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:26.269854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:26.269854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:26.269854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:26.269854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:26.269854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:26.269854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:26.269854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:26.269854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:26.269854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:26.269854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:26.269854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:26.269854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:26.269854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:26.269854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:26.269854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:26.269854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:26.270818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:26.270818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:26.270818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:26.270818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:26.272021 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831cc37-6ce4-4bcf-8826-f8b24b40ebca None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:26.274963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 280/535] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:26.275799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ac417-054b-4e96-a2be-90bbb5e8158a None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:26.309368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4ce5d-ccd1-43e1-8074-d666a12366f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:26.310305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4ce5d-ccd1-43e1-8074-d666a12366f5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:26.310731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4ce5d-ccd1-43e1-8074-d666a12366f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:26.311198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4ce5d-ccd1-43e1-8074-d666a12366f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:26.312167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b4ce5d-ccd1-43e1-8074-d666a12366f5 None admin] RBAC: Adding query filter params (role.id=2a71a120d6e34544aa5a2031ddf65ec8, scope.project.id=56da0975019c44409b53be473131143a, user.id=36de21e0b3de48dabe0f27a9db8a37aa) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:26.312714 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b4ce5d-ccd1-43e1-8074-d666a12366f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:26.314394 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b4ce5d-ccd1-43e1-8074-d666a12366f5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:26.326432 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 256/536] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:23:26 2022] GET /identity/v3/role_assignments?user.id=36de21e0b3de48dabe0f27a9db8a37aa&scope.project.id=56da0975019c44409b53be473131143a&role.id=2a71a120d6e34544aa5a2031ddf65ec8 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:26.327770 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4ce5d-ccd1-43e1-8074-d666a12366f5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:26.380108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:26.421754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ac417-054b-4e96-a2be-90bbb5e8158a None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:26.451605 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b4ce5d-ccd1-43e1-8074-d666a12366f5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:26.480139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e5ea1-99c8-46c5-a091-3aeba73e6d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:26.481087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e5ea1-99c8-46c5-a091-3aeba73e6d1c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:26.481203 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e5ea1-99c8-46c5-a091-3aeba73e6d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:26.481520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e5ea1-99c8-46c5-a091-3aeba73e6d1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:26.482407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e5ea1-99c8-46c5-a091-3aeba73e6d1c None admin] RBAC: Adding query filter params (role.id=2a71a120d6e34544aa5a2031ddf65ec8, scope.project.id=56da0975019c44409b53be473131143a, user.id=95bc067d93fa4207bec1c85600728085) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:26.482733 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e5ea1-99c8-46c5-a091-3aeba73e6d1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:26.484340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e5ea1-99c8-46c5-a091-3aeba73e6d1c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:26.492885 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3e389-8ecd-4352-a303-8be8f0e56257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:26.494139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3e389-8ecd-4352-a303-8be8f0e56257 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:26.494570 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3e389-8ecd-4352-a303-8be8f0e56257 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:26.495049 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3e389-8ecd-4352-a303-8be8f0e56257 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:26.496991 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3e389-8ecd-4352-a303-8be8f0e56257 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:26.497663 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3e389-8ecd-4352-a303-8be8f0e56257 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:26.499547 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 281/537] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:23:26 2022] GET /identity/v3/role_assignments?user.id=95bc067d93fa4207bec1c85600728085&scope.project.id=56da0975019c44409b53be473131143a&role.id=2a71a120d6e34544aa5a2031ddf65ec8 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:26.501226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3e389-8ecd-4352-a303-8be8f0e56257 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:26.515275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 257/538] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:26.524346 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e5ea1-99c8-46c5-a091-3aeba73e6d1c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:26.591656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:26.663856 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e5ea1-99c8-46c5-a091-3aeba73e6d1c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:26.722283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08776e54-7ea7-4093-a387-4ced528d97ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:26.723493 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08776e54-7ea7-4093-a387-4ced528d97ae None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:26.723951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08776e54-7ea7-4093-a387-4ced528d97ae None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:26.724371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08776e54-7ea7-4093-a387-4ced528d97ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:26.739749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08776e54-7ea7-4093-a387-4ced528d97ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:26.741859 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08776e54-7ea7-4093-a387-4ced528d97ae None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:26.762009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-08776e54-7ea7-4093-a387-4ced528d97ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:26.762009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:26.762009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:26.762009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:26.762009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:26.762009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:26.762009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:26.762009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:26.762009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:26.762009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:26.762009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:26.762009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:26.762009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:26.762009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:26.762009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:26.763094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:26.763094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:26.763094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:26.763094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:26.763094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:26.763094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:26.763094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:26.763094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:26.763094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:26.763094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:26.763094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:26.763094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:26.763094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:26.763094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:26.763094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:26.763094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:26.763094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:26.764127 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:26.764127 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:26.764127 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:26.764127 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:26.764127 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:26.764127 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:26.764127 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:26.764127 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:26.766220 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 282/539] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:26.771812 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3e389-8ecd-4352-a303-8be8f0e56257 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:26.917103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:26.924187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3e389-8ecd-4352-a303-8be8f0e56257 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:26.960372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230a2d6-6a1e-4612-b6cc-70ac7abab784 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:26.960692 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230a2d6-6a1e-4612-b6cc-70ac7abab784 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:26.960984 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230a2d6-6a1e-4612-b6cc-70ac7abab784 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:26.962737 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 283/540] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:23:26.967761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb5458-9876-4db9-ac29-f1045309e2b5 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:26.968063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb5458-9876-4db9-ac29-f1045309e2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:26.968356 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb5458-9876-4db9-ac29-f1045309e2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:26.986428 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-98eb5458-9876-4db9-ac29-f1045309e2b5 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:26.987681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-98eb5458-9876-4db9-ac29-f1045309e2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:26.994177 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aafec8-b547-4254-a9aa-ecaae88fcb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:26.995180 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aafec8-b547-4254-a9aa-ecaae88fcb28 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:26.995668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aafec8-b547-4254-a9aa-ecaae88fcb28 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:26.996165 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aafec8-b547-4254-a9aa-ecaae88fcb28 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:26.997177 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aafec8-b547-4254-a9aa-ecaae88fcb28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:26.997690 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aafec8-b547-4254-a9aa-ecaae88fcb28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:26.999350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aafec8-b547-4254-a9aa-ecaae88fcb28 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:27.016722 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 258/541] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:27.019609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aafec8-b547-4254-a9aa-ecaae88fcb28 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:27.139278 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 284/542] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:23:27.158942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb5458-9876-4db9-ac29-f1045309e2b5 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:27.174419 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aafec8-b547-4254-a9aa-ecaae88fcb28 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:27.212307 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74c4a8-baa8-4f01-b67f-d41b8fcb2dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:27.213420 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74c4a8-baa8-4f01-b67f-d41b8fcb2dbf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:27.213912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74c4a8-baa8-4f01-b67f-d41b8fcb2dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:27.214393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74c4a8-baa8-4f01-b67f-d41b8fcb2dbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:27.215468 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74c4a8-baa8-4f01-b67f-d41b8fcb2dbf None admin] RBAC: Adding query filter params (role.id=2a71a120d6e34544aa5a2031ddf65ec8, scope.project.id=56da0975019c44409b53be473131143a, user.id=37ba527b96634fc2a48b6ca9597d8191) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:27.215993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74c4a8-baa8-4f01-b67f-d41b8fcb2dbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:27.217625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74c4a8-baa8-4f01-b67f-d41b8fcb2dbf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:27.224261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 259/543] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:23:27 2022] GET /identity/v3/role_assignments?user.id=37ba527b96634fc2a48b6ca9597d8191&scope.project.id=56da0975019c44409b53be473131143a&role.id=2a71a120d6e34544aa5a2031ddf65ec8 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:27.238301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:27.319446 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb5458-9876-4db9-ac29-f1045309e2b5 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:27.358840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b780beb-911d-42e5-a290-0a7b9d24d239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:27.359709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b780beb-911d-42e5-a290-0a7b9d24d239 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:27.359999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b780beb-911d-42e5-a290-0a7b9d24d239 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:27.360289 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b780beb-911d-42e5-a290-0a7b9d24d239 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:27.373807 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b780beb-911d-42e5-a290-0a7b9d24d239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:27.382624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-0b780beb-911d-42e5-a290-0a7b9d24d239 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:27.382624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:27.382624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:27.382624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:27.382624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:27.382624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:27.382624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:27.382624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:27.382624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:27.382624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:27.382624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:27.382624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:27.382624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:27.382624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:27.382624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:27.382624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:27.383653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:27.383653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:27.383653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:27.383653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:27.383653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:27.383653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:27.383653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:27.383653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:27.383653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:27.383653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:27.383653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:27.383653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:27.383653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:27.383653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:27.383653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:27.383653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:27.384458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:27.384458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:27.384458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:27.384458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:27.384458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:27.384458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:27.384458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:27.384458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:27.384458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:27.384458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:27.384458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:27.384458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:27.384458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 285/544] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:27.391117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74c4a8-baa8-4f01-b67f-d41b8fcb2dbf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:27.448773 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:27.536549 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74c4a8-baa8-4f01-b67f-d41b8fcb2dbf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:27.585602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd80d2ee-516f-4869-888b-5b5fdebfd92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:27.586193 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80d2ee-516f-4869-888b-5b5fdebfd92a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:27.586483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80d2ee-516f-4869-888b-5b5fdebfd92a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:27.586765 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd80d2ee-516f-4869-888b-5b5fdebfd92a None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:27.587544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80d2ee-516f-4869-888b-5b5fdebfd92a None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:27.587852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80d2ee-516f-4869-888b-5b5fdebfd92a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:27.589218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd80d2ee-516f-4869-888b-5b5fdebfd92a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:27.603137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 260/545] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:23:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:27.608593 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b780beb-911d-42e5-a290-0a7b9d24d239 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:27.700025 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679724b-1fa9-4f65-9d9d-8b0e2e3a4f51 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:27.700337 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679724b-1fa9-4f65-9d9d-8b0e2e3a4f51 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:27.700592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679724b-1fa9-4f65-9d9d-8b0e2e3a4f51 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:27.702213 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 261/546] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:27.720854 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289073d8-ffd1-4423-91a3-93f2985f0dd3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:27.721139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289073d8-ffd1-4423-91a3-93f2985f0dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:27.721418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289073d8-ffd1-4423-91a3-93f2985f0dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:27.748331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-289073d8-ffd1-4423-91a3-93f2985f0dd3 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:27.749694 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-289073d8-ffd1-4423-91a3-93f2985f0dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:27.764094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:27.771855 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b780beb-911d-42e5-a290-0a7b9d24d239 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:27.832212 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33148ada-0a55-4242-8acc-5344ea4caa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:27.833019 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33148ada-0a55-4242-8acc-5344ea4caa4b None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:27.833307 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33148ada-0a55-4242-8acc-5344ea4caa4b None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:27.833586 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33148ada-0a55-4242-8acc-5344ea4caa4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:27.856504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33148ada-0a55-4242-8acc-5344ea4caa4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:27.859139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33148ada-0a55-4242-8acc-5344ea4caa4b None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:27.868398 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 262/547] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:23:27.886786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-33148ada-0a55-4242-8acc-5344ea4caa4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:27.886786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:27.886786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:27.886786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:27.886786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:27.886786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:27.886786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:27.886786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:27.886786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:27.886786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:27.886786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:27.886786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:27.886786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:27.886786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:27.886786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:27.887723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:27.887723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:27.887723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:27.887723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:27.887723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:27.887723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:27.887723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:27.887723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:27.887723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:27.887723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:27.887723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:27.887723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:27.887723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:27.887723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:27.887723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:27.887723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:27.887723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:27.888568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:27.888568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:27.888568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:27.888568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:27.888568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 286/548] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:27.893865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289073d8-ffd1-4423-91a3-93f2985f0dd3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:27.909871 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:27.911546 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33148ada-0a55-4242-8acc-5344ea4caa4b None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:28.073016 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289073d8-ffd1-4423-91a3-93f2985f0dd3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:28.097233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33148ada-0a55-4242-8acc-5344ea4caa4b None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:28.140248 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6e014-9d05-43ff-ad85-0ff4da7978c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:28.140929 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6e014-9d05-43ff-ad85-0ff4da7978c6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:28.141221 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6e014-9d05-43ff-ad85-0ff4da7978c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:28.141504 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6e014-9d05-43ff-ad85-0ff4da7978c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:28.142249 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6e014-9d05-43ff-ad85-0ff4da7978c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:28.142568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6e014-9d05-43ff-ad85-0ff4da7978c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:28.143984 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6e014-9d05-43ff-ad85-0ff4da7978c6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:28.159559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579f077-029e-46a5-80b2-f3150c9a1627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:28.160372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579f077-029e-46a5-80b2-f3150c9a1627 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:28.160659 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579f077-029e-46a5-80b2-f3150c9a1627 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:28.160939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579f077-029e-46a5-80b2-f3150c9a1627 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:28.173427 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 263/549] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:28.174337 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6e014-9d05-43ff-ad85-0ff4da7978c6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:28.188392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579f077-029e-46a5-80b2-f3150c9a1627 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:28.196738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-6579f077-029e-46a5-80b2-f3150c9a1627 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:28.196738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:28.196738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:28.196738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:28.196738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:28.196738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:28.196738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:28.196738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:28.196738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:28.196738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:28.196738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:28.196738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:28.196738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:28.196738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:28.196738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:28.196738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:28.198143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:28.198143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:28.198143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:28.198143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:28.198143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:28.198143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:28.198143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:28.198143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:28.198143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:28.198143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:28.198143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:28.198143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:28.198143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:28.198143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:28.198143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:28.198143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:28.199161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:28.199161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:28.199161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:28.199161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:28.199161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:28.199161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:28.199161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:28.199161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:28.199161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:28.199161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:28.199161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:28.199161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:28.199798 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 287/550] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:28.207540 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579f077-029e-46a5-80b2-f3150c9a1627 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:28.230001 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:28.310601 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:28.369813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6e014-9d05-43ff-ad85-0ff4da7978c6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:28.394059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579f077-029e-46a5-80b2-f3150c9a1627 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:28.440712 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d392153-8d1f-4429-97bc-02e0843aaa3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:28.441408 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d392153-8d1f-4429-97bc-02e0843aaa3c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:28.441703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d392153-8d1f-4429-97bc-02e0843aaa3c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:28.441993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d392153-8d1f-4429-97bc-02e0843aaa3c None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:28.442749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d392153-8d1f-4429-97bc-02e0843aaa3c None admin] RBAC: Adding query filter params (name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:28.443094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d392153-8d1f-4429-97bc-02e0843aaa3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:28.443668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e90c41-21e2-4906-9704-2a180b5f7dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:28.444550 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d392153-8d1f-4429-97bc-02e0843aaa3c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:28.444662 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e90c41-21e2-4906-9704-2a180b5f7dde None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:28.444942 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e90c41-21e2-4906-9704-2a180b5f7dde None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:28.445231 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e90c41-21e2-4906-9704-2a180b5f7dde None admin] PATH_INFO: `/v3/users/swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:28.453258 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 288/551] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:23:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:28.457950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e90c41-21e2-4906-9704-2a180b5f7dde None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:28.459490 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e90c41-21e2-4906-9704-2a180b5f7dde None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:28.460793 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d392153-8d1f-4429-97bc-02e0843aaa3c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:28.484187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-32e90c41-21e2-4906-9704-2a180b5f7dde None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:23:28.484187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:28.484187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:28.484187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:28.484187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:28.484187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:28.484187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:28.484187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:28.484187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:28.484187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:28.484187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:28.484187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:28.484187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:28.484187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:28.484187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:28.484187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:28.485161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:28.485161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:28.485161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:28.485161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:28.485161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:28.485161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:28.485161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:28.485161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:28.485161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:28.485161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:28.485161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:28.485161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:28.485161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:28.485161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:28.485161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:28.485161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:28.485161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:28.485161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:28.486283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:28.486283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:28.486283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:28.486283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:28.486283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:28.486283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:28.486283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:28.486283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:28.486283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:23:28.486283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:28.489476 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 264/552] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:23:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:28.496980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e90c41-21e2-4906-9704-2a180b5f7dde None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:28.583366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:28.607682 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d392153-8d1f-4429-97bc-02e0843aaa3c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:28.641838 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91145b48-ef3d-4e51-a4a5-0cf4adc363c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:28.642630 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91145b48-ef3d-4e51-a4a5-0cf4adc363c0 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:28.642939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91145b48-ef3d-4e51-a4a5-0cf4adc363c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:28.643227 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91145b48-ef3d-4e51-a4a5-0cf4adc363c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:28.648983 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e90c41-21e2-4906-9704-2a180b5f7dde None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:28.655032 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91145b48-ef3d-4e51-a4a5-0cf4adc363c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:28.656498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91145b48-ef3d-4e51-a4a5-0cf4adc363c0 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:28.680587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-91145b48-ef3d-4e51-a4a5-0cf4adc363c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:28.680587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:28.680587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:28.680587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:28.680587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:28.680587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:28.680587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:28.680587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:28.680587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:28.680587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:28.680587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:28.680587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:28.680587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:28.680587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:28.680587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:28.681541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:28.681541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:28.681541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:28.681541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:28.681541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:28.681541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:28.681541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:28.681541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:28.681541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:28.681541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:28.681541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:28.681541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:28.681541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:28.681541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:28.681541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:28.681541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:28.681541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:28.682422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:28.682422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:28.682422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:28.682422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:28.682422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 289/553] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:28.687562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91145b48-ef3d-4e51-a4a5-0cf4adc363c0 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:28.744163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:28.759579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08c090-ee4b-464e-b3ca-99b79e31cb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:28.760264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08c090-ee4b-464e-b3ca-99b79e31cb0b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:28.760604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08c090-ee4b-464e-b3ca-99b79e31cb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:28.760892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08c090-ee4b-464e-b3ca-99b79e31cb0b None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:28.761749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08c090-ee4b-464e-b3ca-99b79e31cb0b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:28.762061 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08c090-ee4b-464e-b3ca-99b79e31cb0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:28.763605 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08c090-ee4b-464e-b3ca-99b79e31cb0b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:28.803712 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91145b48-ef3d-4e51-a4a5-0cf4adc363c0 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:28.810570 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 265/554] 10.222.0.43 () {56 vars in 1135 bytes} [Thu Jun 23 22:23:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:28.815897 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08c090-ee4b-464e-b3ca-99b79e31cb0b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:28.828902 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:28.842981 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad83d68-3e53-4916-86ff-e1e269772e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:28.844453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad83d68-3e53-4916-86ff-e1e269772e8e None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:28.844746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad83d68-3e53-4916-86ff-e1e269772e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:28.845034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad83d68-3e53-4916-86ff-e1e269772e8e None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:28.845758 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad83d68-3e53-4916-86ff-e1e269772e8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:28.846071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad83d68-3e53-4916-86ff-e1e269772e8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:28.847466 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad83d68-3e53-4916-86ff-e1e269772e8e None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:28.871399 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 290/555] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:28.876556 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad83d68-3e53-4916-86ff-e1e269772e8e None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:28.996093 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08c090-ee4b-464e-b3ca-99b79e31cb0b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:29.033292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efa01b-f65d-46cb-a5c1-3a7d4effc82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:29.034243 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efa01b-f65d-46cb-a5c1-3a7d4effc82f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:29.034534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efa01b-f65d-46cb-a5c1-3a7d4effc82f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:29.035266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efa01b-f65d-46cb-a5c1-3a7d4effc82f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:29.038369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad83d68-3e53-4916-86ff-e1e269772e8e None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:29.086149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8594d-aa39-49fe-b49e-4a132b61af22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:29.088065 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8594d-aa39-49fe-b49e-4a132b61af22 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:29.088357 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8594d-aa39-49fe-b49e-4a132b61af22 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:29.088643 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8594d-aa39-49fe-b49e-4a132b61af22 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:29.097860 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8594d-aa39-49fe-b49e-4a132b61af22 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:29.099631 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8594d-aa39-49fe-b49e-4a132b61af22 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:29.113155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:29.126545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-e3b8594d-aa39-49fe-b49e-4a132b61af22 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:23:29.126545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:29.126545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:29.126545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:29.126545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:29.126545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:29.126545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:29.126545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:29.126545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:29.126545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:29.126545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:29.126545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:29.126545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:29.126545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:29.126545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:29.126545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:29.129260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:29.129260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:29.129260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:29.129260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:29.129260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:29.129260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:29.129260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:29.129260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:29.129260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:29.129260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:29.129260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:29.129260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:29.129260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:29.129260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:29.129260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:29.129260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:29.129260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:29.129260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:29.131595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:29.131595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:29.131595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:29.131595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:29.131595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:29.131595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:29.131595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:29.131595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:29.131595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:23:29.131595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:29.134300 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efa01b-f65d-46cb-a5c1-3a7d4effc82f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:29.135226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 291/556] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Jun 23 22:23:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:29.135800 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efa01b-f65d-46cb-a5c1-3a7d4effc82f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:29.137034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311ca60-221e-4175-a15a-4bb51791bbaf None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:29.137515 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311ca60-221e-4175-a15a-4bb51791bbaf None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:29.138269 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311ca60-221e-4175-a15a-4bb51791bbaf None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:29.141046 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 292/557] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:29.142010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311ca60-221e-4175-a15a-4bb51791bbaf None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:29.195187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:29.255905 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311ca60-221e-4175-a15a-4bb51791bbaf None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:29.262078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 266/558] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:29.263954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36984ce-337e-4c2e-b4a3-54bdf4ad78a5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:29.264417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36984ce-337e-4c2e-b4a3-54bdf4ad78a5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:29.264825 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36984ce-337e-4c2e-b4a3-54bdf4ad78a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:29.282042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a36984ce-337e-4c2e-b4a3-54bdf4ad78a5 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:29.283263 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a36984ce-337e-4c2e-b4a3-54bdf4ad78a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:29.296827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eeb2b5-1b00-45c1-bc80-b138b11d11f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:29.298007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eeb2b5-1b00-45c1-bc80-b138b11d11f5 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:29.298894 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eeb2b5-1b00-45c1-bc80-b138b11d11f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:29.299270 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eeb2b5-1b00-45c1-bc80-b138b11d11f5 None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:29.300461 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eeb2b5-1b00-45c1-bc80-b138b11d11f5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:29.300985 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eeb2b5-1b00-45c1-bc80-b138b11d11f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:29.302796 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eeb2b5-1b00-45c1-bc80-b138b11d11f5 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:29.332313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 293/559] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Jun 23 22:23:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:29.332313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0929ef6-d3fd-455c-913d-32c5b536a923 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:29.332313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0929ef6-d3fd-455c-913d-32c5b536a923 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:29.332313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0929ef6-d3fd-455c-913d-32c5b536a923 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:29.333211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 294/560] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:29.333484 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0929ef6-d3fd-455c-913d-32c5b536a923 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:29.361028 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 267/561] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:29.362094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36984ce-337e-4c2e-b4a3-54bdf4ad78a5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:29.383658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:29.452377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0929ef6-d3fd-455c-913d-32c5b536a923 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:29.454567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:29.470509 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36984ce-337e-4c2e-b4a3-54bdf4ad78a5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:29.490263 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f2af3-c212-461b-a313-cde87bf6ed81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:29.491722 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f2af3-c212-461b-a313-cde87bf6ed81 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:29.492017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f2af3-c212-461b-a313-cde87bf6ed81 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:29.492296 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f2af3-c212-461b-a313-cde87bf6ed81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:29.503049 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f2af3-c212-461b-a313-cde87bf6ed81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:29.504664 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f2af3-c212-461b-a313-cde87bf6ed81 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:29.506204 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b850bab-eb9b-4f40-b277-d5d2edf9fffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:29.506943 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b850bab-eb9b-4f40-b277-d5d2edf9fffd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:29.507271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b850bab-eb9b-4f40-b277-d5d2edf9fffd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:29.507604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b850bab-eb9b-4f40-b277-d5d2edf9fffd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:29.518339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b850bab-eb9b-4f40-b277-d5d2edf9fffd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:29.519847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b850bab-eb9b-4f40-b277-d5d2edf9fffd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:29.523964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-031f2af3-c212-461b-a313-cde87bf6ed81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:29.523964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:29.523964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:29.523964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:29.523964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:29.523964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:29.523964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:29.523964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:29.523964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:29.523964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:29.523964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:29.523964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:29.523964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:29.523964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:29.523964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:29.524845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:29.524845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:29.524845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:29.524845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:29.524845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:29.524845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:29.524845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:29.524845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:29.524845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:29.524845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:29.524845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:29.524845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:29.524845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:29.524845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:29.524845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:29.524845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:29.524845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:29.525828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:29.525828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:29.525828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:29.525828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:29.525828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 295/562] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:29.530896 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a4a7b-d0f7-4c07-b741-7b6b1f844f8c None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:29.531342 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a4a7b-d0f7-4c07-b741-7b6b1f844f8c None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:29.531597 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a4a7b-d0f7-4c07-b741-7b6b1f844f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:29.539697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-0b850bab-eb9b-4f40-b277-d5d2edf9fffd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:29.539697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:29.539697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:29.539697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:29.539697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:29.539697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:29.539697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:29.539697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:29.539697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:29.539697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:29.539697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:29.539697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:29.539697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:29.539697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:29.539697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:29.540562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:29.540562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:29.540562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:29.540562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:29.540562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:29.540562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:29.540562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:29.540562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:29.540562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:29.540562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:29.540562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:29.540562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:29.540562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:29.540562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:29.540562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:29.540562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:29.540562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:29.546164 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:29.546164 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:29.546164 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:29.546164 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:29.546164 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 268/563] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:29.546164 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b850bab-eb9b-4f40-b277-d5d2edf9fffd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:29.548520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-f37a4a7b-d0f7-4c07-b741-7b6b1f844f8c None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:29.549711 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-f37a4a7b-d0f7-4c07-b741-7b6b1f844f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:29.621778 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:29.631761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 296/564] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:29.633187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a4a7b-d0f7-4c07-b741-7b6b1f844f8c None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:29.655304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b850bab-eb9b-4f40-b277-d5d2edf9fffd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:29.688935 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363364f-b9d0-4bf1-8aef-49d0c5172641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:29.689736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363364f-b9d0-4bf1-8aef-49d0c5172641 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:29.690027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363364f-b9d0-4bf1-8aef-49d0c5172641 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:29.690311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363364f-b9d0-4bf1-8aef-49d0c5172641 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:29.700908 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363364f-b9d0-4bf1-8aef-49d0c5172641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:29.708500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-6363364f-b9d0-4bf1-8aef-49d0c5172641 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:29.708500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:29.708500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:29.708500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:29.708500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:29.708500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:29.708500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:29.708500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:29.708500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:29.708500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:29.708500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:29.708500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:29.708500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:29.708500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:29.708500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:29.708500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:29.710031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:29.710031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:29.710031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:29.710031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:29.710031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:29.710031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:29.710031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:29.710031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:29.710031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:29.710031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:29.710031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:29.710031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:29.710031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:29.710031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:29.710031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:29.710031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:29.710877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:29.710877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:29.710877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:29.710877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:29.710877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:29.710877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:29.710877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:29.710877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:29.710877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:29.710877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:29.710877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:29.710877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:29.710877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 269/565] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:29.711608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363364f-b9d0-4bf1-8aef-49d0c5172641 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:29.724321 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:29.741303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a4a7b-d0f7-4c07-b741-7b6b1f844f8c None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:29.776360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e06ac-14b5-463b-bc23-a8528d8c29dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:29.776963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e06ac-14b5-463b-bc23-a8528d8c29dc None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:29.777253 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e06ac-14b5-463b-bc23-a8528d8c29dc None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:29.777542 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e06ac-14b5-463b-bc23-a8528d8c29dc None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:29.778262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e06ac-14b5-463b-bc23-a8528d8c29dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:29.778577 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e06ac-14b5-463b-bc23-a8528d8c29dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:29.780016 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e06ac-14b5-463b-bc23-a8528d8c29dc None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:29.791763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 297/566] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:29.792554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e06ac-14b5-463b-bc23-a8528d8c29dc None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:29.816041 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363364f-b9d0-4bf1-8aef-49d0c5172641 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:29.845097 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:29.852143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f78a4-2219-45d8-9e8f-bd3eb864fecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:29.852803 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f78a4-2219-45d8-9e8f-bd3eb864fecf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:29.853097 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f78a4-2219-45d8-9e8f-bd3eb864fecf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:29.853384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f78a4-2219-45d8-9e8f-bd3eb864fecf None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:29.854111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f78a4-2219-45d8-9e8f-bd3eb864fecf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:29.854422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f78a4-2219-45d8-9e8f-bd3eb864fecf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:29.855816 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f78a4-2219-45d8-9e8f-bd3eb864fecf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:29.868484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 270/567] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:29.869811 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffed0c-47c4-448a-959e-891fc0d4f3dd None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:29.870104 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffed0c-47c4-448a-959e-891fc0d4f3dd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:29.870390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffed0c-47c4-448a-959e-891fc0d4f3dd None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:29.871799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 271/568] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:29.872345 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffed0c-47c4-448a-959e-891fc0d4f3dd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:29.897024 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e06ac-14b5-463b-bc23-a8528d8c29dc None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:29.931572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e01ef-d50b-46ec-bd9b-ffca2f3aab6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:29.932447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e01ef-d50b-46ec-bd9b-ffca2f3aab6a None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:29.932803 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e01ef-d50b-46ec-bd9b-ffca2f3aab6a None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:29.933151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24e01ef-d50b-46ec-bd9b-ffca2f3aab6a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:29.942857 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24e01ef-d50b-46ec-bd9b-ffca2f3aab6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:29.950496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-e24e01ef-d50b-46ec-bd9b-ffca2f3aab6a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:29.950496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:29.950496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:29.950496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:29.950496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:29.950496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:29.950496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:29.950496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:29.950496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:29.950496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:29.950496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:29.950496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:29.950496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:29.950496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:29.950496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:29.950496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:29.951609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:29.951609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:29.951609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:29.951609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:29.951609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:29.951609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:29.951609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:29.951609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:29.951609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:29.951609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:29.951609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:29.951609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:29.951609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:29.951609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:29.951609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:29.951609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:29.952561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:29.952561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:29.952561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:29.952561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:29.952561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:29.952561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:29.952561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:29.952561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:29.952561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:29.952561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:29.952561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:29.952561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:29.952561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 298/569] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:29.953360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e01ef-d50b-46ec-bd9b-ffca2f3aab6a None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:29.957129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:29.978567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffed0c-47c4-448a-959e-891fc0d4f3dd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:30.011840 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae4fbc-6a86-4a50-bc66-e1517619662d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:30.012500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae4fbc-6a86-4a50-bc66-e1517619662d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:30.012790 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae4fbc-6a86-4a50-bc66-e1517619662d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:30.013079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae4fbc-6a86-4a50-bc66-e1517619662d None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:30.013822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae4fbc-6a86-4a50-bc66-e1517619662d None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:30.014134 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae4fbc-6a86-4a50-bc66-e1517619662d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:30.015524 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae4fbc-6a86-4a50-bc66-e1517619662d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:30.023838 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 272/570] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:23:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:30.024673 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae4fbc-6a86-4a50-bc66-e1517619662d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:30.055821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24e01ef-d50b-46ec-bd9b-ffca2f3aab6a None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:30.078458 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:30.089126 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3071d60-1500-4d0a-81d5-fbf6353ec7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:30.089931 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3071d60-1500-4d0a-81d5-fbf6353ec7be None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:30.090219 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3071d60-1500-4d0a-81d5-fbf6353ec7be None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:30.090501 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3071d60-1500-4d0a-81d5-fbf6353ec7be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:30.097013 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3071d60-1500-4d0a-81d5-fbf6353ec7be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:30.098470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3071d60-1500-4d0a-81d5-fbf6353ec7be None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:30.112702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-d3071d60-1500-4d0a-81d5-fbf6353ec7be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:30.112702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:30.112702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:30.112702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:30.112702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:30.112702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:30.112702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:30.112702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:30.112702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:30.112702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:30.112702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:30.112702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:30.112702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:30.112702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:30.112702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:30.113762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:30.113762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:30.113762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:30.113762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:30.113762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:30.113762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:30.113762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:30.113762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:30.113762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:30.113762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:30.113762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:30.113762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:30.113762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:30.113762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:30.113762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:30.113762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:30.113762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:30.120166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:30.120166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:30.120166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:30.120166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:30.120166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:30.120166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:30.120166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:30.120166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:30.120166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 299/571] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:30.120166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b004dc-36db-4e4a-b195-9d20191a42c7 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:30.120166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b004dc-36db-4e4a-b195-9d20191a42c7 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:30.120893 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b004dc-36db-4e4a-b195-9d20191a42c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:30.123826 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae4fbc-6a86-4a50-bc66-e1517619662d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:30.139458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-d9b004dc-36db-4e4a-b195-9d20191a42c7 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:30.140648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d9b004dc-36db-4e4a-b195-9d20191a42c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:30.158877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c60a20-8701-4152-8f36-aa226706e11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:30.159695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c60a20-8701-4152-8f36-aa226706e11a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:30.159983 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c60a20-8701-4152-8f36-aa226706e11a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:30.160261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c60a20-8701-4152-8f36-aa226706e11a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:30.166865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c60a20-8701-4152-8f36-aa226706e11a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:30.168367 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c60a20-8701-4152-8f36-aa226706e11a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:30.182969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-74c60a20-8701-4152-8f36-aa226706e11a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:30.182969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:30.182969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:30.182969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:30.182969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:30.182969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:30.182969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:30.182969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:30.182969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:30.182969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:30.182969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:30.182969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:30.182969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:30.182969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:30.182969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:30.184012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:30.184012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:30.184012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:30.184012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:30.184012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:30.184012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:30.184012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:30.184012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:30.184012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:30.184012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:30.184012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:30.184012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:30.184012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:30.184012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:30.184012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:30.184012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:30.184012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:30.185103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:30.185103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:30.185103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:30.185103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:30.185103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:30.185103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:30.185103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:30.185103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:30.185103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 273/572] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:30.185103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c60a20-8701-4152-8f36-aa226706e11a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:30.201308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:30.210224 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 300/573] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:30.211056 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b004dc-36db-4e4a-b195-9d20191a42c7 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:30.283671 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c60a20-8701-4152-8f36-aa226706e11a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:30.298607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:30.318976 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32794c73-44ee-4ef8-8a12-2c20e8eef734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:30.319585 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32794c73-44ee-4ef8-8a12-2c20e8eef734 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:30.319876 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32794c73-44ee-4ef8-8a12-2c20e8eef734 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:30.320164 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32794c73-44ee-4ef8-8a12-2c20e8eef734 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:30.320911 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32794c73-44ee-4ef8-8a12-2c20e8eef734 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:30.321224 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32794c73-44ee-4ef8-8a12-2c20e8eef734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:30.322050 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b004dc-36db-4e4a-b195-9d20191a42c7 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:30.322602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32794c73-44ee-4ef8-8a12-2c20e8eef734 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:30.330351 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 274/574] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:23:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:30.331169 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32794c73-44ee-4ef8-8a12-2c20e8eef734 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:30.357154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfeb215-d855-4ade-80c5-ac2add1e8ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:30.357950 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfeb215-d855-4ade-80c5-ac2add1e8ece None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:30.358240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfeb215-d855-4ade-80c5-ac2add1e8ece None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:30.358517 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfeb215-d855-4ade-80c5-ac2add1e8ece None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:30.368616 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfeb215-d855-4ade-80c5-ac2add1e8ece None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:30.370062 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfeb215-d855-4ade-80c5-ac2add1e8ece None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:30.387147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-0bfeb215-d855-4ade-80c5-ac2add1e8ece None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:30.387147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:30.387147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:30.387147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:30.387147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:30.387147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:30.387147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:30.387147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:30.387147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:30.387147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:30.387147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:30.387147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:30.387147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:30.387147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:30.387147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:30.388034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:30.388034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:30.388034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:30.388034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:30.388034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:30.388034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:30.388034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:30.388034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:30.388034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:30.388034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:30.388034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:30.388034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:30.388034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:30.388034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:30.388034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:30.388034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:30.388034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:30.388954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:30.388954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:30.388954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:30.388954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:30.388954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 301/575] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:30.389227 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfeb215-d855-4ade-80c5-ac2add1e8ece None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:30.434956 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32794c73-44ee-4ef8-8a12-2c20e8eef734 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:30.437701 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:30.468753 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af7e4d-cdf8-420d-8c53-e9fd5cbb6ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:30.469423 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af7e4d-cdf8-420d-8c53-e9fd5cbb6ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:30.469713 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af7e4d-cdf8-420d-8c53-e9fd5cbb6ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:30.470001 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af7e4d-cdf8-420d-8c53-e9fd5cbb6ca3 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:30.470799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af7e4d-cdf8-420d-8c53-e9fd5cbb6ca3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:30.471147 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af7e4d-cdf8-420d-8c53-e9fd5cbb6ca3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:30.472610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af7e4d-cdf8-420d-8c53-e9fd5cbb6ca3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:30.485211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 275/576] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:30.485999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af7e4d-cdf8-420d-8c53-e9fd5cbb6ca3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:30.490259 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfeb215-d855-4ade-80c5-ac2add1e8ece None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:30.520859 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:30.525001 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa76519-f950-48c6-898d-be1527aa93c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:30.525660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa76519-f950-48c6-898d-be1527aa93c9 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:30.525944 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa76519-f950-48c6-898d-be1527aa93c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:30.526462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa76519-f950-48c6-898d-be1527aa93c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:30.527027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa76519-f950-48c6-898d-be1527aa93c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:30.527339 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa76519-f950-48c6-898d-be1527aa93c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:30.528761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa76519-f950-48c6-898d-be1527aa93c9 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:30.540402 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 302/577] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:30.541174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa76519-f950-48c6-898d-be1527aa93c9 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:30.586783 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af7e4d-cdf8-420d-8c53-e9fd5cbb6ca3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:30.620987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a17a2-4255-4156-afb0-461c37cc0cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:30.621783 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a17a2-4255-4156-afb0-461c37cc0cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:30.622075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a17a2-4255-4156-afb0-461c37cc0cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:30.622358 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a17a2-4255-4156-afb0-461c37cc0cb9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:30.632961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a17a2-4255-4156-afb0-461c37cc0cb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:30.638617 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa76519-f950-48c6-898d-be1527aa93c9 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:30.640622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-635a17a2-4255-4156-afb0-461c37cc0cb9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:30.640622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:30.640622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:30.640622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:30.640622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:30.640622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:30.640622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:30.640622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:30.640622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:30.640622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:30.640622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:30.640622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:30.640622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:30.640622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:30.640622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:30.640622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:30.641511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:30.641511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:30.641511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:30.641511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:30.641511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:30.641511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:30.641511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:30.641511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:30.641511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:30.641511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:30.641511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:30.641511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:30.641511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:30.641511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:30.641511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:30.641511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:30.642335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:30.642335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:30.642335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:30.642335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:30.642335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:30.642335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:30.642335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:30.642335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:30.642335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:30.642335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:30.642335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:30.642335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:30.642335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 276/578] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:30.643045 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a17a2-4255-4156-afb0-461c37cc0cb9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:30.648592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:30.673178 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640b76f-b556-44b3-8dd4-1e380f29e181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:30.673967 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640b76f-b556-44b3-8dd4-1e380f29e181 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:30.674255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640b76f-b556-44b3-8dd4-1e380f29e181 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:30.674559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640b76f-b556-44b3-8dd4-1e380f29e181 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:30.685680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640b76f-b556-44b3-8dd4-1e380f29e181 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:30.687145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640b76f-b556-44b3-8dd4-1e380f29e181 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:30.704471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-d640b76f-b556-44b3-8dd4-1e380f29e181 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:30.704471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:30.704471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:30.704471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:30.704471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:30.704471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:30.704471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:30.704471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:30.704471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:30.704471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:30.704471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:30.704471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:30.704471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:30.704471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:30.704471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:30.705538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:30.705538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:30.705538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:30.705538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:30.705538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:30.705538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:30.705538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:30.705538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:30.705538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:30.705538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:30.705538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:30.705538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:30.705538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:30.705538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:30.705538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:30.705538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:30.705538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:30.706459 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:30.706459 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:30.706459 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:30.706459 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:30.706459 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 303/579] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:30.706459 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640b76f-b556-44b3-8dd4-1e380f29e181 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:30.743671 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:30.750765 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a17a2-4255-4156-afb0-461c37cc0cb9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:30.785033 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fa246-a700-46fe-9838-6f66dc8e7ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:30.785695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fa246-a700-46fe-9838-6f66dc8e7ecb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:30.785985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fa246-a700-46fe-9838-6f66dc8e7ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:30.786275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fa246-a700-46fe-9838-6f66dc8e7ecb None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:30.787042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fa246-a700-46fe-9838-6f66dc8e7ecb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:30.787411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fa246-a700-46fe-9838-6f66dc8e7ecb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:30.788776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fa246-a700-46fe-9838-6f66dc8e7ecb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:30.800250 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 277/580] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:30.801143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fa246-a700-46fe-9838-6f66dc8e7ecb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:30.804740 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640b76f-b556-44b3-8dd4-1e380f29e181 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:30.838988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135d20d-c8ee-44ae-a3b5-6ce3abc48433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:30.839580 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135d20d-c8ee-44ae-a3b5-6ce3abc48433 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:30.839865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135d20d-c8ee-44ae-a3b5-6ce3abc48433 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:30.840144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135d20d-c8ee-44ae-a3b5-6ce3abc48433 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:30.840965 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135d20d-c8ee-44ae-a3b5-6ce3abc48433 None admin] RBAC: Adding query filter params (role.id=2a71a120d6e34544aa5a2031ddf65ec8, scope.project.id=56da0975019c44409b53be473131143a, user.id=8ce35c43e8294f4cbc880f2219a6eb76) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:30.841279 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135d20d-c8ee-44ae-a3b5-6ce3abc48433 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:30.842712 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c135d20d-c8ee-44ae-a3b5-6ce3abc48433 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:30.848090 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 304/581] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:23:30 2022] GET /identity/v3/role_assignments?user.id=8ce35c43e8294f4cbc880f2219a6eb76&scope.project.id=56da0975019c44409b53be473131143a&role.id=2a71a120d6e34544aa5a2031ddf65ec8 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:30.848911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135d20d-c8ee-44ae-a3b5-6ce3abc48433 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:30.877745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:30.928372 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fa246-a700-46fe-9838-6f66dc8e7ecb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:30.966546 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae521396-c9a3-4224-8277-8c8d29c088c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:30.967510 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae521396-c9a3-4224-8277-8c8d29c088c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:30.967685 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135d20d-c8ee-44ae-a3b5-6ce3abc48433 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:30.967815 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae521396-c9a3-4224-8277-8c8d29c088c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:30.968078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae521396-c9a3-4224-8277-8c8d29c088c1 None admin] PATH_INFO: `/v3/projects/56da0975019c44409b53be473131143a/users/35d686923ef541819de3c77847d38511/roles/2a71a120d6e34544aa5a2031ddf65ec8` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:30.986808 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae521396-c9a3-4224-8277-8c8d29c088c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=56da0975019c44409b53be473131143a, user_id=35d686923ef541819de3c77847d38511, role_id=2a71a120d6e34544aa5a2031ddf65ec8)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:30.988357 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae521396-c9a3-4224-8277-8c8d29c088c1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:31.010964 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 278/582] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:23:30 2022] PUT /identity/v3/projects/56da0975019c44409b53be473131143a/users/35d686923ef541819de3c77847d38511/roles/2a71a120d6e34544aa5a2031ddf65ec8 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:23:31.010964 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae521396-c9a3-4224-8277-8c8d29c088c1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:31.013720 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e955129-6ba2-4099-bb12-e897141eeb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:31.014509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e955129-6ba2-4099-bb12-e897141eeb25 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:31.014869 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e955129-6ba2-4099-bb12-e897141eeb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:31.015249 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e955129-6ba2-4099-bb12-e897141eeb25 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:31.016073 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e955129-6ba2-4099-bb12-e897141eeb25 None admin] RBAC: Adding query filter params (name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:31.016391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e955129-6ba2-4099-bb12-e897141eeb25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:31.017768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e955129-6ba2-4099-bb12-e897141eeb25 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:31.023565 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:31.032225 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 305/583] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:23:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:31.033170 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e955129-6ba2-4099-bb12-e897141eeb25 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:31.143271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:31.158656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae521396-c9a3-4224-8277-8c8d29c088c1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:31.164198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e955129-6ba2-4099-bb12-e897141eeb25 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:31.203206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39d6e6-6d0b-4f93-a05d-0c0f283aa32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:31.204396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39d6e6-6d0b-4f93-a05d-0c0f283aa32f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:31.204396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39d6e6-6d0b-4f93-a05d-0c0f283aa32f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:31.204562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39d6e6-6d0b-4f93-a05d-0c0f283aa32f None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:31.205259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39d6e6-6d0b-4f93-a05d-0c0f283aa32f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:31.205572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39d6e6-6d0b-4f93-a05d-0c0f283aa32f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:31.206423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d13b37-2bc9-4847-b118-6b3f35cd5785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:31.206968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39d6e6-6d0b-4f93-a05d-0c0f283aa32f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:31.207288 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d13b37-2bc9-4847-b118-6b3f35cd5785 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:31.207582 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d13b37-2bc9-4847-b118-6b3f35cd5785 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:31.207869 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d13b37-2bc9-4847-b118-6b3f35cd5785 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:31.216842 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d13b37-2bc9-4847-b118-6b3f35cd5785 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:31.218296 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d13b37-2bc9-4847-b118-6b3f35cd5785 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:31.222091 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 279/584] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:31.223647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39d6e6-6d0b-4f93-a05d-0c0f283aa32f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:31.238691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-e2d13b37-2bc9-4847-b118-6b3f35cd5785 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 22:23:31.238691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:31.238691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:31.238691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:31.238691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:31.238691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:31.238691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:31.238691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:31.238691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:31.238691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:31.238691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:31.238691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:31.238691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:31.238691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:31.238691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:31.238691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:31.239754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:31.239754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:31.239754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:31.239754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:31.239754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:31.239754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:31.239754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:31.239754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:31.239754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:31.239754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:31.239754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:31.239754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:31.239754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:31.239754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:31.239754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:31.239754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:31.239754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:31.239754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:31.240806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:31.240806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:31.240806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:31.240806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:31.240806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:31.240806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:31.240806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:31.240806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:31.240806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 22:23:31.240806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:31.240806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 306/585] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:31.241392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d13b37-2bc9-4847-b118-6b3f35cd5785 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:31.319149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:31.361712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39d6e6-6d0b-4f93-a05d-0c0f283aa32f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:31.372243 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d13b37-2bc9-4847-b118-6b3f35cd5785 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:31.404014 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155b792-6a1e-4c1a-a3fd-deb8f0b277e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:31.404492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155b792-6a1e-4c1a-a3fd-deb8f0b277e1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:31.404823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155b792-6a1e-4c1a-a3fd-deb8f0b277e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:31.405064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155b792-6a1e-4c1a-a3fd-deb8f0b277e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:31.409908 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8d6b3-6042-4d70-9d73-ff23108c3365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:31.410950 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8d6b3-6042-4d70-9d73-ff23108c3365 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:31.411422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8d6b3-6042-4d70-9d73-ff23108c3365 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:31.411884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8d6b3-6042-4d70-9d73-ff23108c3365 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:31.417227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155b792-6a1e-4c1a-a3fd-deb8f0b277e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:31.418682 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155b792-6a1e-4c1a-a3fd-deb8f0b277e1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:31.421915 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8d6b3-6042-4d70-9d73-ff23108c3365 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:31.423597 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8d6b3-6042-4d70-9d73-ff23108c3365 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:31.438563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f155b792-6a1e-4c1a-a3fd-deb8f0b277e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:31.438563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:31.438563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:31.438563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:31.438563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:31.438563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:31.438563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:31.438563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:31.438563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:31.438563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:31.438563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:31.438563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:31.438563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:31.438563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:31.438563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:31.439556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:31.439556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:31.439556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:31.439556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:31.439556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:31.439556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:31.439556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:31.439556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:31.439556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:31.439556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:31.439556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:31.439556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:31.439556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:31.439556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:31.439556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:31.439556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:31.439556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:31.440453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:31.440453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:31.440453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:31.440453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:31.441824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-a8c8d6b3-6042-4d70-9d73-ff23108c3365 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 22:23:31.441824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:31.441824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:31.441824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:31.441824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:31.441824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:31.441824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:31.441824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:31.441824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:31.441824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:31.441824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:31.441824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:31.441824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:31.441824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:31.441824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:31.441824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:31.442696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:31.442696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:31.442696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:31.442696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:31.442696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:31.442696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:31.442696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:31.442696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:31.442696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:31.442696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:31.442696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:31.442696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:31.442696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:31.442696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:31.442696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:31.442696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:31.442696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:31.442696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:31.443905 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:31.443905 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:31.443905 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:31.443905 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:31.443905 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:31.443905 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:31.443905 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:31.443905 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:31.443905 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 22:23:31.443905 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:31.444996 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 280/586] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:31.445873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155b792-6a1e-4c1a-a3fd-deb8f0b277e1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:31.446837 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 307/587] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:31.447842 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8d6b3-6042-4d70-9d73-ff23108c3365 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:31.473745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:31.555334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155b792-6a1e-4c1a-a3fd-deb8f0b277e1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:31.561450 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8d6b3-6042-4d70-9d73-ff23108c3365 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:31.573582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:31.592003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e4013-473e-454f-8e3a-d64f3792638e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:31.592752 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e4013-473e-454f-8e3a-d64f3792638e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:31.592958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e4013-473e-454f-8e3a-d64f3792638e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:31.593240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e4013-473e-454f-8e3a-d64f3792638e None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:31.594097 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e4013-473e-454f-8e3a-d64f3792638e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:31.594414 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e4013-473e-454f-8e3a-d64f3792638e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:31.596066 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e4013-473e-454f-8e3a-d64f3792638e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:31.601603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bc50d-83d3-4ed0-ac8a-aca0c6a380e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:31.602563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bc50d-83d3-4ed0-ac8a-aca0c6a380e1 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:31.603107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bc50d-83d3-4ed0-ac8a-aca0c6a380e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:31.603613 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bc50d-83d3-4ed0-ac8a-aca0c6a380e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:31.604557 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7bc50d-83d3-4ed0-ac8a-aca0c6a380e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:31.605065 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7bc50d-83d3-4ed0-ac8a-aca0c6a380e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:31.606939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7bc50d-83d3-4ed0-ac8a-aca0c6a380e1 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:31.614777 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 281/588] 10.222.0.43 () {56 vars in 1137 bytes} [Thu Jun 23 22:23:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:31.616903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e4013-473e-454f-8e3a-d64f3792638e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:31.621332 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 308/589] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:31.622386 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bc50d-83d3-4ed0-ac8a-aca0c6a380e1 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:31.693067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:31.734462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bc50d-83d3-4ed0-ac8a-aca0c6a380e1 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:31.739771 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e4013-473e-454f-8e3a-d64f3792638e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:31.770879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1aa24-2f4e-4d8e-94bc-7d2fbf33a01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:31.773111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1aa24-2f4e-4d8e-94bc-7d2fbf33a01f None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:31.773650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1aa24-2f4e-4d8e-94bc-7d2fbf33a01f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:31.774151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb1aa24-2f4e-4d8e-94bc-7d2fbf33a01f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:31.780640 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1fcf4-6353-422f-89e1-43b63b9d8960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:31.781376 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1fcf4-6353-422f-89e1-43b63b9d8960 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:31.781721 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1fcf4-6353-422f-89e1-43b63b9d8960 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:31.782011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1fcf4-6353-422f-89e1-43b63b9d8960 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:31.783053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1fcf4-6353-422f-89e1-43b63b9d8960 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:31.783375 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1fcf4-6353-422f-89e1-43b63b9d8960 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:31.785032 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1fcf4-6353-422f-89e1-43b63b9d8960 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:31.787115 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1aa24-2f4e-4d8e-94bc-7d2fbf33a01f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:31.789530 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb1aa24-2f4e-4d8e-94bc-7d2fbf33a01f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:31.805457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 282/590] 10.222.0.43 () {56 vars in 1137 bytes} [Thu Jun 23 22:23:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:31.806414 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1fcf4-6353-422f-89e1-43b63b9d8960 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:31.808999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-9fb1aa24-2f4e-4d8e-94bc-7d2fbf33a01f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:31.808999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:31.808999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:31.808999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:31.808999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:31.808999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:31.808999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:31.808999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:31.808999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:31.808999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:31.808999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:31.808999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:31.808999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:31.808999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:31.808999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:31.810166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:31.810166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:31.810166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:31.810166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:31.810166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:31.810166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:31.810166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:31.810166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:31.810166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:31.810166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:31.810166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:31.810166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:31.810166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:31.810166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:31.810166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:31.810166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:31.810166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:31.811083 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:31.811083 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:31.811083 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:31.811083 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:31.813232 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 309/591] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:31.814235 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1aa24-2f4e-4d8e-94bc-7d2fbf33a01f None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:31.845292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:31.920953 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1fcf4-6353-422f-89e1-43b63b9d8960 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:31.923895 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb1aa24-2f4e-4d8e-94bc-7d2fbf33a01f None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:31.941846 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:31.960348 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e169ae-ee73-449d-9e61-88cbcdc91f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:31.961138 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e169ae-ee73-449d-9e61-88cbcdc91f46 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:31.961434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e169ae-ee73-449d-9e61-88cbcdc91f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:31.961717 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e169ae-ee73-449d-9e61-88cbcdc91f46 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:31.963757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c68dd-6f0b-4399-9c1c-d2a6d1cf8070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:31.965070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c68dd-6f0b-4399-9c1c-d2a6d1cf8070 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:31.965582 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c68dd-6f0b-4399-9c1c-d2a6d1cf8070 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:31.966062 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c68dd-6f0b-4399-9c1c-d2a6d1cf8070 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:31.970421 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e169ae-ee73-449d-9e61-88cbcdc91f46 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:31.972127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e169ae-ee73-449d-9e61-88cbcdc91f46 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:31.978366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c68dd-6f0b-4399-9c1c-d2a6d1cf8070 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:31.980106 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c68dd-6f0b-4399-9c1c-d2a6d1cf8070 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:31.988418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d2e169ae-ee73-449d-9e61-88cbcdc91f46 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 22:23:31.988418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:31.988418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:31.988418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:31.988418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:31.988418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:31.988418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:31.988418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:31.988418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:31.988418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:31.988418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:31.988418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:31.988418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:31.988418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:31.988418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:31.988418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:31.989362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:31.989362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:31.989362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:31.989362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:31.989362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:31.989362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:31.989362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:31.989362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:31.989362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:31.989362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:31.989362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:31.989362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:31.989362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:31.989362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:31.989362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:31.989362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:31.989362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:31.989362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:31.990335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:31.990335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:31.990335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:31.990335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:31.990335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:31.990335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:31.990335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:31.990335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:31.990335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 22:23:31.990335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:31.995169 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 283/592] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:31.996053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e169ae-ee73-449d-9e61-88cbcdc91f46 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:32.002262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-9a0c68dd-6f0b-4399-9c1c-d2a6d1cf8070 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:32.002262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:32.002262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:32.002262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:32.002262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:32.002262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:32.002262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:32.002262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:32.002262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:32.002262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:32.002262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:32.002262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:32.002262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:32.002262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:32.002262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:32.003157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:32.003157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:32.003157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:32.003157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:32.003157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:32.003157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:32.003157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:32.003157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:32.003157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:32.003157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:32.003157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:32.003157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:32.003157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:32.003157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:32.003157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:32.003157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:32.003157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:32.004053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:32.004053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:32.004053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:32.004053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:32.006759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 310/593] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:32.009579 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c68dd-6f0b-4399-9c1c-d2a6d1cf8070 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:32.104658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:32.111154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e169ae-ee73-449d-9e61-88cbcdc91f46 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:32.125619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c68dd-6f0b-4399-9c1c-d2a6d1cf8070 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:32.149757 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942ff17-162a-4156-a8cd-7cd5573d51cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:32.150434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942ff17-162a-4156-a8cd-7cd5573d51cc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:32.150723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942ff17-162a-4156-a8cd-7cd5573d51cc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:32.151036 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942ff17-162a-4156-a8cd-7cd5573d51cc None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:32.151787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942ff17-162a-4156-a8cd-7cd5573d51cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:32.152347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942ff17-162a-4156-a8cd-7cd5573d51cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:32.153446 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942ff17-162a-4156-a8cd-7cd5573d51cc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:32.166330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 284/594] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:32.167287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942ff17-162a-4156-a8cd-7cd5573d51cc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:32.168106 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d68ed-35fc-47b6-bade-2fee05f7f22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:32.169044 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d68ed-35fc-47b6-bade-2fee05f7f22d None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:32.169538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d68ed-35fc-47b6-bade-2fee05f7f22d None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:32.170105 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d68ed-35fc-47b6-bade-2fee05f7f22d None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:32.171613 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d68ed-35fc-47b6-bade-2fee05f7f22d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:32.172167 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d68ed-35fc-47b6-bade-2fee05f7f22d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:32.173874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d68ed-35fc-47b6-bade-2fee05f7f22d None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:32.196109 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 311/595] 10.222.0.43 () {56 vars in 1139 bytes} [Thu Jun 23 22:23:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:32.197428 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d68ed-35fc-47b6-bade-2fee05f7f22d None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:32.236273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:32.302697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942ff17-162a-4156-a8cd-7cd5573d51cc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:32.338691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d68ed-35fc-47b6-bade-2fee05f7f22d None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:32.341038 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960abe34-e287-4256-a54a-88ce75bc6b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:32.342228 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960abe34-e287-4256-a54a-88ce75bc6b96 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:32.342228 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960abe34-e287-4256-a54a-88ce75bc6b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:32.342403 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960abe34-e287-4256-a54a-88ce75bc6b96 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:32.343105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960abe34-e287-4256-a54a-88ce75bc6b96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:32.343417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960abe34-e287-4256-a54a-88ce75bc6b96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:32.344828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960abe34-e287-4256-a54a-88ce75bc6b96 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:32.357995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 285/596] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:32.358547 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960abe34-e287-4256-a54a-88ce75bc6b96 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:32.375691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:32.388011 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caae3bb-d41e-4243-80fd-2360017af7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:32.389331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caae3bb-d41e-4243-80fd-2360017af7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:32.390066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caae3bb-d41e-4243-80fd-2360017af7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:32.390975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caae3bb-d41e-4243-80fd-2360017af7b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:32.401067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caae3bb-d41e-4243-80fd-2360017af7b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:32.402787 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caae3bb-d41e-4243-80fd-2360017af7b5 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:32.422650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-3caae3bb-d41e-4243-80fd-2360017af7b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:32.422650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:32.422650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:32.422650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:32.422650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:32.422650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:32.422650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:32.422650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:32.422650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:32.422650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:32.422650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:32.422650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:32.422650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:32.422650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:32.422650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:32.423619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:32.423619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:32.423619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:32.423619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:32.423619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:32.423619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:32.423619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:32.423619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:32.423619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:32.423619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:32.423619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:32.423619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:32.423619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:32.423619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:32.423619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:32.423619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:32.423619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:32.424506 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:32.424506 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:32.424506 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:32.424506 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:32.424506 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:32.424506 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:32.424506 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:32.424506 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:32.427166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 312/597] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:32.429587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caae3bb-d41e-4243-80fd-2360017af7b5 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:32.471511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960abe34-e287-4256-a54a-88ce75bc6b96 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:32.509256 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96008184-c013-4569-ab2d-3b8d14eb2c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:32.510128 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96008184-c013-4569-ab2d-3b8d14eb2c2e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:32.510515 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96008184-c013-4569-ab2d-3b8d14eb2c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:32.510893 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96008184-c013-4569-ab2d-3b8d14eb2c2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:32.521527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96008184-c013-4569-ab2d-3b8d14eb2c2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:32.523999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96008184-c013-4569-ab2d-3b8d14eb2c2e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:32.542488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-96008184-c013-4569-ab2d-3b8d14eb2c2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:32.542488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:32.542488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:32.542488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:32.542488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:32.542488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:32.542488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:32.542488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:32.542488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:32.542488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:32.542488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:32.542488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:32.542488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:32.542488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:32.542488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:32.543730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:32.543730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:32.543730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:32.543730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:32.543730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:32.543730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:32.543730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:32.543730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:32.543730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:32.543730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:32.543730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:32.543730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:32.543730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:32.543730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:32.543730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:32.543730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:32.543730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:32.544818 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:32.544818 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:32.544818 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:32.544818 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:32.549596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:32.551027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 286/598] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:32.552189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96008184-c013-4569-ab2d-3b8d14eb2c2e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:32.571538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caae3bb-d41e-4243-80fd-2360017af7b5 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:32.619654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a9ca8-0445-4da9-a9be-af1bc78d64c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:32.620989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a9ca8-0445-4da9-a9be-af1bc78d64c0 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:32.621430 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a9ca8-0445-4da9-a9be-af1bc78d64c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:32.622110 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a9ca8-0445-4da9-a9be-af1bc78d64c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:32.626427 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:32.634230 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a9ca8-0445-4da9-a9be-af1bc78d64c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:32.636130 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a9ca8-0445-4da9-a9be-af1bc78d64c0 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:32.656779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-c97a9ca8-0445-4da9-a9be-af1bc78d64c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:32.656779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:32.656779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:32.656779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:32.656779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:32.656779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:32.656779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:32.656779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:32.656779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:32.656779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:32.656779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:32.656779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:32.656779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:32.656779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:32.656779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:32.657911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:32.657911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:32.657911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:32.657911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:32.657911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:32.657911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:32.657911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:32.657911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:32.657911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:32.657911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:32.657911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:32.657911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:32.657911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:32.657911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:32.657911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:32.657911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:32.657911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:32.659031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:32.659031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:32.659031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:32.659031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:32.659031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:32.659031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:32.659031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:32.659031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:32.661696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 313/599] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:32.663561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a9ca8-0445-4da9-a9be-af1bc78d64c0 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:32.669187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96008184-c013-4569-ab2d-3b8d14eb2c2e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:32.705384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366c225-3f47-4e15-abe5-15208486c393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:32.706459 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366c225-3f47-4e15-abe5-15208486c393 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:32.706582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366c225-3f47-4e15-abe5-15208486c393 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:32.706996 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366c225-3f47-4e15-abe5-15208486c393 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:32.708069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0366c225-3f47-4e15-abe5-15208486c393 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:32.708490 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0366c225-3f47-4e15-abe5-15208486c393 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:32.710136 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0366c225-3f47-4e15-abe5-15208486c393 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:32.723028 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 287/600] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:32.723886 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366c225-3f47-4e15-abe5-15208486c393 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:32.797410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a9ca8-0445-4da9-a9be-af1bc78d64c0 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:32.834407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17738fd4-3b51-439b-8183-1f3c1e83ed97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:32.835290 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17738fd4-3b51-439b-8183-1f3c1e83ed97 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:32.835687 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17738fd4-3b51-439b-8183-1f3c1e83ed97 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:32.836077 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17738fd4-3b51-439b-8183-1f3c1e83ed97 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:32.836915 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17738fd4-3b51-439b-8183-1f3c1e83ed97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:32.837606 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17738fd4-3b51-439b-8183-1f3c1e83ed97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:32.838834 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17738fd4-3b51-439b-8183-1f3c1e83ed97 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:32.853088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366c225-3f47-4e15-abe5-15208486c393 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:32.856481 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 314/601] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:32.857554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17738fd4-3b51-439b-8183-1f3c1e83ed97 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:32.869945 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:32.889400 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:32.902015 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50aef9e-cc63-47b0-952f-5f9479be5f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:32.902913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50aef9e-cc63-47b0-952f-5f9479be5f3e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:32.903378 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50aef9e-cc63-47b0-952f-5f9479be5f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:32.903812 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50aef9e-cc63-47b0-952f-5f9479be5f3e None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:32.904799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50aef9e-cc63-47b0-952f-5f9479be5f3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:32.905260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50aef9e-cc63-47b0-952f-5f9479be5f3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:32.907277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50aef9e-cc63-47b0-952f-5f9479be5f3e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:32.923945 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 288/602] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:32.925394 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50aef9e-cc63-47b0-952f-5f9479be5f3e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:32.985271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17738fd4-3b51-439b-8183-1f3c1e83ed97 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:33.024442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb5384-120b-47cf-9930-ff89ac3fe47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:33.025467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb5384-120b-47cf-9930-ff89ac3fe47f None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:33.025764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb5384-120b-47cf-9930-ff89ac3fe47f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:33.026046 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb5384-120b-47cf-9930-ff89ac3fe47f None admin] PATH_INFO: `/v3/projects/56da0975019c44409b53be473131143a/users/36de21e0b3de48dabe0f27a9db8a37aa/roles/2a71a120d6e34544aa5a2031ddf65ec8` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:33.041900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb5384-120b-47cf-9930-ff89ac3fe47f None admin] RBAC: Authorizing `identity:create_grant(project_id=56da0975019c44409b53be473131143a, user_id=36de21e0b3de48dabe0f27a9db8a37aa, role_id=2a71a120d6e34544aa5a2031ddf65ec8)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:33.043392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb5384-120b-47cf-9930-ff89ac3fe47f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:33.051902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50aef9e-cc63-47b0-952f-5f9479be5f3e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:33.061209 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 315/603] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:23:32 2022] PUT /identity/v3/projects/56da0975019c44409b53be473131143a/users/36de21e0b3de48dabe0f27a9db8a37aa/roles/2a71a120d6e34544aa5a2031ddf65ec8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:23:33.062060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb5384-120b-47cf-9930-ff89ac3fe47f None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:33.108637 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91538443-5350-4fd1-a7a4-529bd1d6d44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:33.110166 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91538443-5350-4fd1-a7a4-529bd1d6d44d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:33.110676 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91538443-5350-4fd1-a7a4-529bd1d6d44d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:33.111161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91538443-5350-4fd1-a7a4-529bd1d6d44d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:33.129383 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91538443-5350-4fd1-a7a4-529bd1d6d44d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:33.132593 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91538443-5350-4fd1-a7a4-529bd1d6d44d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:33.148397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-91538443-5350-4fd1-a7a4-529bd1d6d44d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:33.148397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:33.148397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:33.148397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:33.148397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:33.148397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:33.148397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:33.148397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:33.148397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:33.148397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:33.148397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:33.148397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:33.148397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:33.148397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:33.148397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:33.149542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:33.149542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:33.149542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:33.149542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:33.149542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:33.149542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:33.149542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:33.149542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:33.149542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:33.149542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:33.149542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:33.149542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:33.149542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:33.149542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:33.149542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:33.149542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:33.149542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:33.150567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:33.150567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:33.150567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:33.150567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:33.150567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:33.150567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:33.150567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:33.150567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:33.150567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 289/604] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:33.151449 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a58189-a618-4964-b2e0-6b4af7f65598 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:33.151736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a58189-a618-4964-b2e0-6b4af7f65598 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:33.151948 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a58189-a618-4964-b2e0-6b4af7f65598 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:33.157696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 290/605] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:33.157696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839f418-293c-4ef3-b079-affb6f875550 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:33.157696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839f418-293c-4ef3-b079-affb6f875550 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:33.157696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839f418-293c-4ef3-b079-affb6f875550 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:33.157696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 291/606] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:33.157696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839f418-293c-4ef3-b079-affb6f875550 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:33.164864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:33.210795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:33.225301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb5384-120b-47cf-9930-ff89ac3fe47f None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:33.265685 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd7c0e-f383-49a9-857d-668c56086077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:33.266619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd7c0e-f383-49a9-857d-668c56086077 None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:33.266912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd7c0e-f383-49a9-857d-668c56086077 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:33.267224 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd7c0e-f383-49a9-857d-668c56086077 None admin] PATH_INFO: `/v3/projects/56da0975019c44409b53be473131143a/users/95bc067d93fa4207bec1c85600728085/roles/2a71a120d6e34544aa5a2031ddf65ec8` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:33.274237 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839f418-293c-4ef3-b079-affb6f875550 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:33.284062 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd7c0e-f383-49a9-857d-668c56086077 None admin] RBAC: Authorizing `identity:create_grant(project_id=56da0975019c44409b53be473131143a, user_id=95bc067d93fa4207bec1c85600728085, role_id=2a71a120d6e34544aa5a2031ddf65ec8)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:33.285612 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd7c0e-f383-49a9-857d-668c56086077 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:33.306668 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 316/607] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:23:33 2022] PUT /identity/v3/projects/56da0975019c44409b53be473131143a/users/95bc067d93fa4207bec1c85600728085/roles/2a71a120d6e34544aa5a2031ddf65ec8 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:23:33.308208 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fce6eb-5de7-4d4b-bd12-8bf0c7881f81 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:33.308503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fce6eb-5de7-4d4b-bd12-8bf0c7881f81 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:33.308786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fce6eb-5de7-4d4b-bd12-8bf0c7881f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:33.314768 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85bdfe8-2726-45f4-8898-10ffed4f8e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:33.317012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85bdfe8-2726-45f4-8898-10ffed4f8e41 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:33.317534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85bdfe8-2726-45f4-8898-10ffed4f8e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:33.317998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85bdfe8-2726-45f4-8898-10ffed4f8e41 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:33.320057 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85bdfe8-2726-45f4-8898-10ffed4f8e41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:33.320635 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85bdfe8-2726-45f4-8898-10ffed4f8e41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:33.324687 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85bdfe8-2726-45f4-8898-10ffed4f8e41 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:33.328115 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-43fce6eb-5de7-4d4b-bd12-8bf0c7881f81 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:33.329439 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-43fce6eb-5de7-4d4b-bd12-8bf0c7881f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:33.343699 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 292/608] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:33.345687 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39221a01-cc9a-426c-92e1-6d702dc8ed01 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:33.346464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39221a01-cc9a-426c-92e1-6d702dc8ed01 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:33.346968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39221a01-cc9a-426c-92e1-6d702dc8ed01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:33.373791 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-39221a01-cc9a-426c-92e1-6d702dc8ed01 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:33.375719 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-39221a01-cc9a-426c-92e1-6d702dc8ed01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:33.424897 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 317/609] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:33.424897 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fce6eb-5de7-4d4b-bd12-8bf0c7881f81 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:33.480172 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:33.481506 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 293/610] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:33.482631 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39221a01-cc9a-426c-92e1-6d702dc8ed01 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:33.525985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:33.563407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fce6eb-5de7-4d4b-bd12-8bf0c7881f81 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:33.598199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39221a01-cc9a-426c-92e1-6d702dc8ed01 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:33.601196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb840815-c3ae-4c56-b87d-c38d4d5323e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:33.602111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb840815-c3ae-4c56-b87d-c38d4d5323e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:33.602403 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb840815-c3ae-4c56-b87d-c38d4d5323e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:33.602684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb840815-c3ae-4c56-b87d-c38d4d5323e3 None admin] PATH_INFO: `/v3/projects/56da0975019c44409b53be473131143a/users/37ba527b96634fc2a48b6ca9597d8191/roles/2a71a120d6e34544aa5a2031ddf65ec8` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:33.618758 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb840815-c3ae-4c56-b87d-c38d4d5323e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=56da0975019c44409b53be473131143a, user_id=37ba527b96634fc2a48b6ca9597d8191, role_id=2a71a120d6e34544aa5a2031ddf65ec8)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:33.620265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb840815-c3ae-4c56-b87d-c38d4d5323e3 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:33.636775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 318/611] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:23:33 2022] PUT /identity/v3/projects/56da0975019c44409b53be473131143a/users/37ba527b96634fc2a48b6ca9597d8191/roles/2a71a120d6e34544aa5a2031ddf65ec8 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:23:33.637180 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eed556-ceb3-4b63-b592-510a3ce713d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:33.638218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eed556-ceb3-4b63-b592-510a3ce713d3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:33.638750 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eed556-ceb3-4b63-b592-510a3ce713d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:33.639248 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eed556-ceb3-4b63-b592-510a3ce713d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:33.644726 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb840815-c3ae-4c56-b87d-c38d4d5323e3 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:33.663491 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eed556-ceb3-4b63-b592-510a3ce713d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:33.675913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a4eed556-ceb3-4b63-b592-510a3ce713d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:33.675913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:33.675913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:33.675913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:33.675913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:33.675913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:33.675913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:33.675913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:33.675913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:33.675913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:33.675913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:33.675913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:33.675913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:33.675913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:33.675913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:33.675913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:33.676917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:33.676917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:33.676917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:33.676917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:33.676917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:33.676917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:33.676917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:33.676917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:33.676917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:33.676917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:33.676917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:33.676917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:33.676917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:33.676917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:33.676917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:33.676917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:33.677743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:33.677743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:33.677743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:33.677743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:33.677743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:33.677743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:33.677743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:33.677743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:33.677743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:33.677743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:33.677743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:33.677743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:33.680660 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 294/612] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:33.686426 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eed556-ceb3-4b63-b592-510a3ce713d3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:33.832392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:33.853416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb840815-c3ae-4c56-b87d-c38d4d5323e3 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:33.869982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:33.877496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eed556-ceb3-4b63-b592-510a3ce713d3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:33.891973 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ca2ad-ae03-444c-ba8e-201e22648478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:33.893099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ca2ad-ae03-444c-ba8e-201e22648478 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:33.893607 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ca2ad-ae03-444c-ba8e-201e22648478 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:33.894075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ca2ad-ae03-444c-ba8e-201e22648478 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:33.907436 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ca2ad-ae03-444c-ba8e-201e22648478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:33.915429 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146247d-7d34-457e-baa1-84b4e24d2d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:33.916101 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146247d-7d34-457e-baa1-84b4e24d2d59 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:33.916392 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146247d-7d34-457e-baa1-84b4e24d2d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:33.916678 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146247d-7d34-457e-baa1-84b4e24d2d59 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:33.917272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-6a2ca2ad-ae03-444c-ba8e-201e22648478 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:33.917272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:33.917272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:33.917272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:33.917272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:33.917272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:33.917272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:33.917272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:33.917272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:33.917272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:33.917272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:33.917272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:33.917272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:33.917272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:33.917272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:33.917272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:33.918154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:33.918154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:33.918154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:33.918154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:33.918154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:33.918154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:33.918154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:33.918154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:33.918154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:33.918154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:33.918154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:33.918154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:33.918154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:33.918154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:33.918154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:33.918154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:33.919026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:33.919026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:33.919026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:33.919026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:33.919026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:33.919026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:33.919026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:33.919026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:33.919026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:33.919026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:33.919026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:33.919026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:33.919660 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2146247d-7d34-457e-baa1-84b4e24d2d59 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:33.919660 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2146247d-7d34-457e-baa1-84b4e24d2d59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:33.919660 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2146247d-7d34-457e-baa1-84b4e24d2d59 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:33.921887 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 319/613] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:33.932585 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ca2ad-ae03-444c-ba8e-201e22648478 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:33.935381 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 295/614] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:23:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:33.943971 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146247d-7d34-457e-baa1-84b4e24d2d59 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:34.069521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ca2ad-ae03-444c-ba8e-201e22648478 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:34.143477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34fa260-e410-4311-a3e5-0cdc0b61a58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:34.144170 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fa260-e410-4311-a3e5-0cdc0b61a58f None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:34.144457 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fa260-e410-4311-a3e5-0cdc0b61a58f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:34.144739 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fa260-e410-4311-a3e5-0cdc0b61a58f None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:34.145498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34fa260-e410-4311-a3e5-0cdc0b61a58f None admin] RBAC: Adding query filter params (name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:34.145811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34fa260-e410-4311-a3e5-0cdc0b61a58f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:34.148130 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34fa260-e410-4311-a3e5-0cdc0b61a58f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:34.164713 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146247d-7d34-457e-baa1-84b4e24d2d59 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:34.168139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 320/615] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:23:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:34.174525 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34fa260-e410-4311-a3e5-0cdc0b61a58f None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:34.190439 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:34.217264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644438f7-c5dd-4977-a92b-9a81e7357172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:34.218063 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644438f7-c5dd-4977-a92b-9a81e7357172 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:34.218352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644438f7-c5dd-4977-a92b-9a81e7357172 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:34.218636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644438f7-c5dd-4977-a92b-9a81e7357172 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:34.234507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644438f7-c5dd-4977-a92b-9a81e7357172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:34.236146 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644438f7-c5dd-4977-a92b-9a81e7357172 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:34.267917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-644438f7-c5dd-4977-a92b-9a81e7357172 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:34.267917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:34.267917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:34.267917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:34.267917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:34.267917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:34.267917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:34.267917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:34.267917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:34.267917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:34.267917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:34.267917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:34.267917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:34.267917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:34.267917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:34.268858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:34.268858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:34.268858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:34.268858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:34.268858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:34.268858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:34.268858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:34.268858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:34.268858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:34.268858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:34.268858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:34.268858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:34.268858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:34.268858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:34.268858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:34.268858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:34.268858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:34.269741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:34.269741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:34.269741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:34.269741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:34.278534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 296/616] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:34.286489 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644438f7-c5dd-4977-a92b-9a81e7357172 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:34.308001 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:34.371560 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34fa260-e410-4311-a3e5-0cdc0b61a58f None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:34.434179 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763f562-b6b2-4c8f-9a07-4b76270208d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:34.436589 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763f562-b6b2-4c8f-9a07-4b76270208d2 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:34.437275 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763f562-b6b2-4c8f-9a07-4b76270208d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:34.438397 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763f562-b6b2-4c8f-9a07-4b76270208d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:34.457175 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763f562-b6b2-4c8f-9a07-4b76270208d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:34.460110 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763f562-b6b2-4c8f-9a07-4b76270208d2 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:34.473458 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644438f7-c5dd-4977-a92b-9a81e7357172 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:34.490775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-1763f562-b6b2-4c8f-9a07-4b76270208d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:34.490775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:34.490775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:34.490775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:34.490775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:34.490775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:34.490775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:34.490775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:34.490775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:34.490775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:34.490775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:34.490775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:34.490775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:34.490775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:34.490775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:34.491805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:34.491805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:34.491805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:34.491805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:34.491805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:34.491805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:34.491805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:34.491805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:34.491805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:34.491805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:34.491805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:34.491805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:34.491805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:34.491805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:34.491805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:34.491805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:34.491805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:34.492726 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:34.492726 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:34.492726 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:34.492726 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:34.501821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 321/617] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:23:34.508658 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763f562-b6b2-4c8f-9a07-4b76270208d2 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:34.540292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a756e5-860d-48f7-8f2b-dd7181edde9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:34.541299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a756e5-860d-48f7-8f2b-dd7181edde9a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:34.541798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a756e5-860d-48f7-8f2b-dd7181edde9a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:34.542278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a756e5-860d-48f7-8f2b-dd7181edde9a None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:34.543236 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a756e5-860d-48f7-8f2b-dd7181edde9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:34.544983 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a756e5-860d-48f7-8f2b-dd7181edde9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:34.546548 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a756e5-860d-48f7-8f2b-dd7181edde9a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:34.570809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 297/618] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:34.576646 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a756e5-860d-48f7-8f2b-dd7181edde9a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:34.627314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:34.678905 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763f562-b6b2-4c8f-9a07-4b76270208d2 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:34.716133 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:34.738404 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3479a11-97ae-454a-b7a0-73a02a7ee971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:34.739109 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3479a11-97ae-454a-b7a0-73a02a7ee971 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:34.739400 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3479a11-97ae-454a-b7a0-73a02a7ee971 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:34.739682 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3479a11-97ae-454a-b7a0-73a02a7ee971 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:34.741000 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3479a11-97ae-454a-b7a0-73a02a7ee971 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:34.741521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3479a11-97ae-454a-b7a0-73a02a7ee971 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:34.742939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3479a11-97ae-454a-b7a0-73a02a7ee971 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:34.769383 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 322/619] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:34.770003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3479a11-97ae-454a-b7a0-73a02a7ee971 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:34.778717 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a756e5-860d-48f7-8f2b-dd7181edde9a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:34.832655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d77c9-32fa-4e64-943f-492aa387c667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:34.833576 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d77c9-32fa-4e64-943f-492aa387c667 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:34.833866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d77c9-32fa-4e64-943f-492aa387c667 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:34.834711 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d77c9-32fa-4e64-943f-492aa387c667 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:34.843501 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d77c9-32fa-4e64-943f-492aa387c667 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:34.844938 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d77c9-32fa-4e64-943f-492aa387c667 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:34.860259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5e9d77c9-32fa-4e64-943f-492aa387c667 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:23:34.860259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:34.860259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:34.860259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:34.860259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:34.860259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:34.860259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:34.860259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:34.860259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:34.860259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:34.860259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:34.860259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:34.860259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:34.860259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:34.860259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:34.860259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:34.861410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:34.861410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:34.861410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:34.861410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:34.861410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:34.861410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:34.861410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:34.861410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:34.861410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:34.861410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:34.861410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:34.861410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:34.861410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:34.861410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:34.861410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:34.861410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:34.861410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:34.861410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:34.862887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:34.862887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:34.862887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:34.862887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:34.862887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:34.862887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:34.862887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:34.862887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:34.862887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:23:34.862887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:34.862887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 298/620] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:23:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:23:34.867053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d77c9-32fa-4e64-943f-492aa387c667 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:34.957362 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3479a11-97ae-454a-b7a0-73a02a7ee971 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:35.003365 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:35.021361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50ad22-373e-496e-83dd-bf5a43da257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:35.022907 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50ad22-373e-496e-83dd-bf5a43da257d None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:35.024577 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50ad22-373e-496e-83dd-bf5a43da257d None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:35.026091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50ad22-373e-496e-83dd-bf5a43da257d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:35.036757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50ad22-373e-496e-83dd-bf5a43da257d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:35.040971 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50ad22-373e-496e-83dd-bf5a43da257d None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:35.046145 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d77c9-32fa-4e64-943f-492aa387c667 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:35.067027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-2d50ad22-373e-496e-83dd-bf5a43da257d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:23:35.067027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:35.067027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:35.067027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:35.067027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:35.067027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:35.067027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:35.067027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:35.067027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:35.067027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:35.067027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:35.067027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:35.067027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:35.067027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:35.067027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:35.067027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:35.068374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:35.068374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:35.068374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:35.068374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:35.068374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:35.068374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:35.068374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:35.068374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:35.068374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:35.068374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:35.068374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:35.068374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:35.068374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:35.068374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:35.068374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:35.068374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:35.068374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:35.068374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:35.069874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:35.069874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:35.069874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:35.069874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:35.069874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:35.069874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:35.069874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:35.069874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:35.069874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:23:35.069874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:35.069874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 323/621] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Jun 23 22:23:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:35.076024 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50ad22-373e-496e-83dd-bf5a43da257d None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:35.095298 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:35.111650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd9a94-4115-46af-b3cb-eef3bc31bdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:35.112326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd9a94-4115-46af-b3cb-eef3bc31bdf1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:35.112613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd9a94-4115-46af-b3cb-eef3bc31bdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:35.112890 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd9a94-4115-46af-b3cb-eef3bc31bdf1 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:35.113741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd9a94-4115-46af-b3cb-eef3bc31bdf1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:35.114051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd9a94-4115-46af-b3cb-eef3bc31bdf1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:35.115574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd9a94-4115-46af-b3cb-eef3bc31bdf1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:35.155106 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 299/622] 10.222.0.43 () {56 vars in 1135 bytes} [Thu Jun 23 22:23:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:35.156009 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd9a94-4115-46af-b3cb-eef3bc31bdf1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:35.266985 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50ad22-373e-496e-83dd-bf5a43da257d None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:35.347034 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd9a94-4115-46af-b3cb-eef3bc31bdf1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:35.364011 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d8ae5-db4f-4c65-8c62-31ca7807e707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:35.366941 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d8ae5-db4f-4c65-8c62-31ca7807e707 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:35.366941 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d8ae5-db4f-4c65-8c62-31ca7807e707 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:35.366941 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d8ae5-db4f-4c65-8c62-31ca7807e707 None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:35.367198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d8ae5-db4f-4c65-8c62-31ca7807e707 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:35.367469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d8ae5-db4f-4c65-8c62-31ca7807e707 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:35.374941 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d8ae5-db4f-4c65-8c62-31ca7807e707 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:35.395909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 324/623] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Jun 23 22:23:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:35.406662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ff817-c802-4244-a72c-df48104b6359 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:35.406662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ff817-c802-4244-a72c-df48104b6359 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:35.406662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ff817-c802-4244-a72c-df48104b6359 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:35.406662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 325/624] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:35 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:35.407955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ff817-c802-4244-a72c-df48104b6359 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:35.414943 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66398bb7-d831-4fa8-81c8-eb8cbb00f76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:35.414943 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66398bb7-d831-4fa8-81c8-eb8cbb00f76d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:35.414943 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66398bb7-d831-4fa8-81c8-eb8cbb00f76d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:35.414943 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66398bb7-d831-4fa8-81c8-eb8cbb00f76d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:35.438947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66398bb7-d831-4fa8-81c8-eb8cbb00f76d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:35.439152 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66398bb7-d831-4fa8-81c8-eb8cbb00f76d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:35.470954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-66398bb7-d831-4fa8-81c8-eb8cbb00f76d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:35.470954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:35.470954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:35.470954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:35.470954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:35.470954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:35.470954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:35.470954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:35.470954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:35.470954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:35.470954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:35.470954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:35.470954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:35.470954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:35.470954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:35.471924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:35.471924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:35.471924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:35.471924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:35.471924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:35.471924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:35.471924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:35.471924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:35.471924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:35.471924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:35.471924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:35.471924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:35.471924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:35.471924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:35.471924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:35.471924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:35.471924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:35.472799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:35.472799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:35.472799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:35.472799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:35.477174 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 300/625] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:35.480366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381be4f7-5ca3-4004-a2e2-518993599409 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:35.480366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381be4f7-5ca3-4004-a2e2-518993599409 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:35.480366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381be4f7-5ca3-4004-a2e2-518993599409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:35.493499 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:35.523038 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-381be4f7-5ca3-4004-a2e2-518993599409 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:35.526948 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-381be4f7-5ca3-4004-a2e2-518993599409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:35.561058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ff817-c802-4244-a72c-df48104b6359 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:35.606947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d93d1-dc94-416b-bf0e-7dcc2960b540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:35.606947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d93d1-dc94-416b-bf0e-7dcc2960b540 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:35.606947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d93d1-dc94-416b-bf0e-7dcc2960b540 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:35.606947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d93d1-dc94-416b-bf0e-7dcc2960b540 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:35.618951 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:35.687009 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 301/626] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:35.687945 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381be4f7-5ca3-4004-a2e2-518993599409 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:35.713939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d93d1-dc94-416b-bf0e-7dcc2960b540 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:35.716978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d93d1-dc94-416b-bf0e-7dcc2960b540 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:35.773200 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:35.839086 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381be4f7-5ca3-4004-a2e2-518993599409 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:35.845226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 326/627] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:35.845226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d93d1-dc94-416b-bf0e-7dcc2960b540 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:35.883109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c4435-8b9d-4919-9082-6d460cc99e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:35.883780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c4435-8b9d-4919-9082-6d460cc99e37 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:35.884072 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c4435-8b9d-4919-9082-6d460cc99e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:35.884359 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c4435-8b9d-4919-9082-6d460cc99e37 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:35.885206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c4435-8b9d-4919-9082-6d460cc99e37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:35.885520 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c4435-8b9d-4919-9082-6d460cc99e37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:35.886911 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c4435-8b9d-4919-9082-6d460cc99e37 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:35.899396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 302/628] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:35.900961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400923f-e0e2-4ad4-b469-dc83960157e6 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:35.901317 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400923f-e0e2-4ad4-b469-dc83960157e6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:35.901669 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400923f-e0e2-4ad4-b469-dc83960157e6 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:35.906863 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 303/629] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:35.906863 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400923f-e0e2-4ad4-b469-dc83960157e6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:35.935090 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:35.957012 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d93d1-dc94-416b-bf0e-7dcc2960b540 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:35.995289 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee627e76-afbd-4a8f-8a22-63cf9a9263e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:35.996108 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee627e76-afbd-4a8f-8a22-63cf9a9263e7 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:35.996374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee627e76-afbd-4a8f-8a22-63cf9a9263e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:35.996654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee627e76-afbd-4a8f-8a22-63cf9a9263e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:36.004433 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:36.009442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee627e76-afbd-4a8f-8a22-63cf9a9263e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:36.011340 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400923f-e0e2-4ad4-b469-dc83960157e6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:36.018037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-ee627e76-afbd-4a8f-8a22-63cf9a9263e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:36.018037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:36.018037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:36.018037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:36.018037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:36.018037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:36.018037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:36.018037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:36.018037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:36.018037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:36.018037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:36.018037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:36.018037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:36.018037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:36.018037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:36.018037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:36.019028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:36.019028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:36.019028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:36.019028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:36.019028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:36.019028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:36.019028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:36.019028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:36.019028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:36.019028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:36.019028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:36.019028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:36.019028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:36.019028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:36.019028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:36.019028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:36.019877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:36.019877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:36.019877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:36.019877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:36.019877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:36.019877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:36.019877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:36.019877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:36.019877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:36.019877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:36.019877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:36.019877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:36.019877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 327/630] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:36.020578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee627e76-afbd-4a8f-8a22-63cf9a9263e7 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:36.054122 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489c45e-84d7-4fe8-a924-650c0d9b3230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:36.054910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489c45e-84d7-4fe8-a924-650c0d9b3230 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:36.055161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489c45e-84d7-4fe8-a924-650c0d9b3230 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:36.055444 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489c45e-84d7-4fe8-a924-650c0d9b3230 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:36.065960 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489c45e-84d7-4fe8-a924-650c0d9b3230 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:36.067454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489c45e-84d7-4fe8-a924-650c0d9b3230 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:36.085608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a489c45e-84d7-4fe8-a924-650c0d9b3230 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:36.085608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:36.085608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:36.085608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:36.085608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:36.085608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:36.085608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:36.085608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:36.085608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:36.085608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:36.085608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:36.085608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:36.085608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:36.085608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:36.085608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:36.086496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:36.086496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:36.086496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:36.086496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:36.086496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:36.086496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:36.086496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:36.086496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:36.086496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:36.086496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:36.086496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:36.086496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:36.086496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:36.086496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:36.086496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:36.086496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:36.086496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:36.087449 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:36.087449 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:36.087449 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:36.087449 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:36.087449 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 304/631] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:36.088002 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfd282-6f80-4a54-9a06-63ec7797fe11 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:36.088262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfd282-6f80-4a54-9a06-63ec7797fe11 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:36.088542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfd282-6f80-4a54-9a06-63ec7797fe11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:36.104857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9ccfd282-6f80-4a54-9a06-63ec7797fe11 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:36.106050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9ccfd282-6f80-4a54-9a06-63ec7797fe11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:36.136079 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee627e76-afbd-4a8f-8a22-63cf9a9263e7 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:36.170634 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe15b9e-62c9-4a6d-ad46-cc344159556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:36.171470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe15b9e-62c9-4a6d-ad46-cc344159556a None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:36.171762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe15b9e-62c9-4a6d-ad46-cc344159556a None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:36.172048 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe15b9e-62c9-4a6d-ad46-cc344159556a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:36.177130 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 305/632] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:36.178593 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfd282-6f80-4a54-9a06-63ec7797fe11 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:36.179541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe15b9e-62c9-4a6d-ad46-cc344159556a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:36.180997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe15b9e-62c9-4a6d-ad46-cc344159556a None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:36.196124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-abe15b9e-62c9-4a6d-ad46-cc344159556a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:36.196124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:36.196124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:36.196124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:36.196124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:36.196124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:36.196124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:36.196124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:36.196124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:36.196124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:36.196124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:36.196124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:36.196124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:36.196124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:36.196124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:36.197189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:36.197189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:36.197189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:36.197189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:36.197189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:36.197189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:36.197189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:36.197189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:36.197189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:36.197189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:36.197189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:36.197189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:36.197189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:36.197189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:36.197189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:36.197189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:36.197189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:36.198078 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:36.198078 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:36.198078 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:36.198078 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:36.198078 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:36.198078 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:36.198078 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:36.198078 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:36.198078 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 328/633] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:36.198580 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363be85-f6f3-4b45-a4a6-1212604fce32 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:36.198647 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363be85-f6f3-4b45-a4a6-1212604fce32 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:36.199180 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363be85-f6f3-4b45-a4a6-1212604fce32 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:36.200434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 329/634] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:36.201149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363be85-f6f3-4b45-a4a6-1212604fce32 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:36.202791 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:36.261877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:36.289703 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfd282-6f80-4a54-9a06-63ec7797fe11 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:36.304068 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363be85-f6f3-4b45-a4a6-1212604fce32 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:36.322273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df3479-80b0-492f-8f9c-4e84af682291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:36.322877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df3479-80b0-492f-8f9c-4e84af682291 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:36.323198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df3479-80b0-492f-8f9c-4e84af682291 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:36.323482 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df3479-80b0-492f-8f9c-4e84af682291 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:36.324229 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df3479-80b0-492f-8f9c-4e84af682291 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:36.324540 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df3479-80b0-492f-8f9c-4e84af682291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:36.325909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df3479-80b0-492f-8f9c-4e84af682291 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:36.333756 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 306/635] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:23:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:36.335716 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df3479-80b0-492f-8f9c-4e84af682291 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:36.343639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92158e2-7a60-4d84-8206-4f0081339e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:36.344312 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92158e2-7a60-4d84-8206-4f0081339e1c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:36.344602 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92158e2-7a60-4d84-8206-4f0081339e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:36.344884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92158e2-7a60-4d84-8206-4f0081339e1c None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:36.345608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92158e2-7a60-4d84-8206-4f0081339e1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:36.345922 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92158e2-7a60-4d84-8206-4f0081339e1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:36.347327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92158e2-7a60-4d84-8206-4f0081339e1c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:36.358636 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 330/636] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:36.359451 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92158e2-7a60-4d84-8206-4f0081339e1c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:36.412036 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:36.436959 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df3479-80b0-492f-8f9c-4e84af682291 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:36.461102 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92158e2-7a60-4d84-8206-4f0081339e1c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:36.474851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a346860-dcb2-45de-9a22-6173cd4eca06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:36.475546 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a346860-dcb2-45de-9a22-6173cd4eca06 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:36.475831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a346860-dcb2-45de-9a22-6173cd4eca06 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:36.476112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a346860-dcb2-45de-9a22-6173cd4eca06 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:36.476898 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a346860-dcb2-45de-9a22-6173cd4eca06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:36.477209 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a346860-dcb2-45de-9a22-6173cd4eca06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:36.478220 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:36.478633 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a346860-dcb2-45de-9a22-6173cd4eca06 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:36.491395 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 307/637] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:36.492606 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f785b-1cd5-473a-93d5-219dac1a0ba8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:36.492897 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f785b-1cd5-473a-93d5-219dac1a0ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:36.493176 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f785b-1cd5-473a-93d5-219dac1a0ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:36.501388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227431e3-eb0c-43ea-aafe-69749e8b3f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:36.502320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227431e3-eb0c-43ea-aafe-69749e8b3f60 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:36.502725 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227431e3-eb0c-43ea-aafe-69749e8b3f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:36.503153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227431e3-eb0c-43ea-aafe-69749e8b3f60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:36.509380 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9b7f785b-1cd5-473a-93d5-219dac1a0ba8 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:36.510554 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9b7f785b-1cd5-473a-93d5-219dac1a0ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:36.514146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227431e3-eb0c-43ea-aafe-69749e8b3f60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:36.522696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-227431e3-eb0c-43ea-aafe-69749e8b3f60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:36.522696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:36.522696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:36.522696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:36.522696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:36.522696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:36.522696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:36.522696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:36.522696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:36.522696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:36.522696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:36.522696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:36.522696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:36.522696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:36.522696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:36.522696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:36.523827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:36.523827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:36.523827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:36.523827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:36.523827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:36.523827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:36.523827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:36.523827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:36.523827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:36.523827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:36.523827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:36.523827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:36.523827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:36.523827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:36.523827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:36.523827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:36.524795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:36.524795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:36.524795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:36.524795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:36.524795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:36.524795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:36.524795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:36.524795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:36.524795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:36.524795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:36.524795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:36.524795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:36.524795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 331/638] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:36.525578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227431e3-eb0c-43ea-aafe-69749e8b3f60 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:36.581063 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 308/639] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:36.581828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f785b-1cd5-473a-93d5-219dac1a0ba8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:36.622270 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227431e3-eb0c-43ea-aafe-69749e8b3f60 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:36.640859 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:36.656289 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c12d4-ae31-4cea-be9c-c3eb06bb7b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:36.657087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c12d4-ae31-4cea-be9c-c3eb06bb7b33 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:36.657384 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c12d4-ae31-4cea-be9c-c3eb06bb7b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:36.657663 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c12d4-ae31-4cea-be9c-c3eb06bb7b33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:36.668752 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17c12d4-ae31-4cea-be9c-c3eb06bb7b33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:36.670203 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17c12d4-ae31-4cea-be9c-c3eb06bb7b33 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:36.679698 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f785b-1cd5-473a-93d5-219dac1a0ba8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:36.686765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-c17c12d4-ae31-4cea-be9c-c3eb06bb7b33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:36.686765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:36.686765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:36.686765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:36.686765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:36.686765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:36.686765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:36.686765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:36.686765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:36.686765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:36.686765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:36.686765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:36.686765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:36.686765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:36.686765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:36.687654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:36.687654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:36.687654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:36.687654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:36.687654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:36.687654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:36.687654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:36.687654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:36.687654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:36.687654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:36.687654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:36.687654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:36.687654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:36.687654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:36.687654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:36.687654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:36.687654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:36.688533 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:36.688533 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:36.688533 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:36.688533 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:36.688533 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 332/640] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:36.688813 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c12d4-ae31-4cea-be9c-c3eb06bb7b33 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:36.718453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d12f5f-e86a-4abc-b8d0-9ce6527b4768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:36.719154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:36.719268 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d12f5f-e86a-4abc-b8d0-9ce6527b4768 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:36.719548 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d12f5f-e86a-4abc-b8d0-9ce6527b4768 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:36.719835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d12f5f-e86a-4abc-b8d0-9ce6527b4768 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:36.727021 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d12f5f-e86a-4abc-b8d0-9ce6527b4768 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:36.728528 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d12f5f-e86a-4abc-b8d0-9ce6527b4768 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:36.742004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c0d12f5f-e86a-4abc-b8d0-9ce6527b4768 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:36.742004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:36.742004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:36.742004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:36.742004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:36.742004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:36.742004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:36.742004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:36.742004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:36.742004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:36.742004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:36.742004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:36.742004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:36.742004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:36.742004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:36.743157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:36.743157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:36.743157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:36.743157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:36.743157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:36.743157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:36.743157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:36.743157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:36.743157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:36.743157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:36.743157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:36.743157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:36.743157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:36.743157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:36.743157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:36.743157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:36.743157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:36.744257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:36.744257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:36.744257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:36.744257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:36.744257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:36.744257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:36.744257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:36.744257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:36.744257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 309/641] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:36.744257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d12f5f-e86a-4abc-b8d0-9ce6527b4768 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:36.795304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c12d4-ae31-4cea-be9c-c3eb06bb7b33 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:36.830830 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18896979-f4ff-4636-9ebf-607912ec27cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:36.831822 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18896979-f4ff-4636-9ebf-607912ec27cc None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:36.832053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18896979-f4ff-4636-9ebf-607912ec27cc None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:36.832337 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18896979-f4ff-4636-9ebf-607912ec27cc None admin] PATH_INFO: `/v3/projects/56da0975019c44409b53be473131143a/users/8ce35c43e8294f4cbc880f2219a6eb76/roles/2a71a120d6e34544aa5a2031ddf65ec8` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:36.845221 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d12f5f-e86a-4abc-b8d0-9ce6527b4768 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:36.847427 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18896979-f4ff-4636-9ebf-607912ec27cc None admin] RBAC: Authorizing `identity:create_grant(project_id=56da0975019c44409b53be473131143a, user_id=8ce35c43e8294f4cbc880f2219a6eb76, role_id=2a71a120d6e34544aa5a2031ddf65ec8)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:36.848881 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18896979-f4ff-4636-9ebf-607912ec27cc None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:36.862274 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 333/642] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:23:36 2022] PUT /identity/v3/projects/56da0975019c44409b53be473131143a/users/8ce35c43e8294f4cbc880f2219a6eb76/roles/2a71a120d6e34544aa5a2031ddf65ec8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:23:36.867868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18896979-f4ff-4636-9ebf-607912ec27cc None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:36.870475 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:36.885213 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc07f2e-a2c9-4ff5-b124-707afeccb708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:36.886032 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc07f2e-a2c9-4ff5-b124-707afeccb708 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:36.886322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc07f2e-a2c9-4ff5-b124-707afeccb708 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:36.886608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc07f2e-a2c9-4ff5-b124-707afeccb708 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:36.887421 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc07f2e-a2c9-4ff5-b124-707afeccb708 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:36.887737 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc07f2e-a2c9-4ff5-b124-707afeccb708 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:36.889402 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc07f2e-a2c9-4ff5-b124-707afeccb708 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:36.900777 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 310/643] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:23:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:36.901833 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc07f2e-a2c9-4ff5-b124-707afeccb708 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:36.967189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:36.989443 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18896979-f4ff-4636-9ebf-607912ec27cc None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:37.027961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc07f2e-a2c9-4ff5-b124-707afeccb708 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:37.032848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cef5ea-d627-4eaa-acd2-2cb14b3966d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:37.033694 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cef5ea-d627-4eaa-acd2-2cb14b3966d2 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:37.033982 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cef5ea-d627-4eaa-acd2-2cb14b3966d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:37.034263 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cef5ea-d627-4eaa-acd2-2cb14b3966d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:37.048256 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cef5ea-d627-4eaa-acd2-2cb14b3966d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:37.058583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-73cef5ea-d627-4eaa-acd2-2cb14b3966d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:37.058583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:37.058583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:37.058583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:37.058583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:37.058583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:37.058583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:37.058583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:37.058583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:37.058583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:37.058583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:37.058583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:37.058583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:37.058583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:37.058583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:37.058583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:37.059595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:37.059595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:37.059595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:37.059595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:37.059595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:37.059595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:37.059595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:37.059595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:37.059595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:37.059595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:37.059595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:37.059595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:37.059595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:37.059595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:37.059595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:37.059595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:37.060512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:37.060512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:37.060512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:37.060512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:37.060512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:37.060512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:37.060512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:37.060512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:37.060512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:37.060512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:37.060512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:37.060512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:37.060512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 334/644] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:37.061344 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cef5ea-d627-4eaa-acd2-2cb14b3966d2 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:37.069283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71a67b-e7a6-4a03-a3cd-1cfbdeaef4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:37.070048 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71a67b-e7a6-4a03-a3cd-1cfbdeaef4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:37.070725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71a67b-e7a6-4a03-a3cd-1cfbdeaef4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:37.070725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71a67b-e7a6-4a03-a3cd-1cfbdeaef4f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:37.071385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71a67b-e7a6-4a03-a3cd-1cfbdeaef4f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:37.071693 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71a67b-e7a6-4a03-a3cd-1cfbdeaef4f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:37.073072 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71a67b-e7a6-4a03-a3cd-1cfbdeaef4f5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:37.086062 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 311/645] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:37.087814 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71a67b-e7a6-4a03-a3cd-1cfbdeaef4f5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:37.146401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:37.177315 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cef5ea-d627-4eaa-acd2-2cb14b3966d2 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:37.206029 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71a67b-e7a6-4a03-a3cd-1cfbdeaef4f5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:37.213622 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83360d69-eaa0-48f5-a04f-fbfd3d1e6b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:37.214288 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83360d69-eaa0-48f5-a04f-fbfd3d1e6b14 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:37.214580 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83360d69-eaa0-48f5-a04f-fbfd3d1e6b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:37.214860 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83360d69-eaa0-48f5-a04f-fbfd3d1e6b14 None admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:37.215681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83360d69-eaa0-48f5-a04f-fbfd3d1e6b14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:37.215992 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83360d69-eaa0-48f5-a04f-fbfd3d1e6b14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:37.217425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83360d69-eaa0-48f5-a04f-fbfd3d1e6b14 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:37.226993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:37.230833 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 335/646] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:37.231834 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83360d69-eaa0-48f5-a04f-fbfd3d1e6b14 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:37.249199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfda8a-91f3-4c04-917c-e00eb0ecb07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:37.249892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfda8a-91f3-4c04-917c-e00eb0ecb07e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:37.250178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfda8a-91f3-4c04-917c-e00eb0ecb07e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:37.250460 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfda8a-91f3-4c04-917c-e00eb0ecb07e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:37.261209 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cfda8a-91f3-4c04-917c-e00eb0ecb07e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:37.262639 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cfda8a-91f3-4c04-917c-e00eb0ecb07e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:37.280987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-62cfda8a-91f3-4c04-917c-e00eb0ecb07e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:37.280987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:37.280987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:37.280987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:37.280987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:37.280987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:37.280987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:37.280987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:37.280987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:37.280987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:37.280987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:37.280987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:37.280987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:37.280987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:37.280987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:37.281925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:37.281925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:37.281925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:37.281925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:37.281925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:37.281925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:37.281925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:37.281925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:37.281925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:37.281925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:37.281925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:37.281925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:37.281925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:37.281925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:37.281925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:37.281925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:37.281925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:37.282830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:37.282830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:37.282830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:37.282830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:37.282830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 312/647] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:37.283149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfda8a-91f3-4c04-917c-e00eb0ecb07e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:37.344247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83360d69-eaa0-48f5-a04f-fbfd3d1e6b14 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:37.380752 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b68816-89e1-4813-bf80-835c58d1e937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:37.381964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b68816-89e1-4813-bf80-835c58d1e937 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:37.382241 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b68816-89e1-4813-bf80-835c58d1e937 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:37.382526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b68816-89e1-4813-bf80-835c58d1e937 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:37.382981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:37.383307 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b68816-89e1-4813-bf80-835c58d1e937 None admin] RBAC: Adding query filter params (name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:37.383619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b68816-89e1-4813-bf80-835c58d1e937 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:37.385002 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b68816-89e1-4813-bf80-835c58d1e937 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:37.388867 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfda8a-91f3-4c04-917c-e00eb0ecb07e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:37.398639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 336/648] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:23:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:37.399413 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b68816-89e1-4813-bf80-835c58d1e937 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:37.424002 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8efbf5-71ee-41aa-800d-98e3ad28289a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:37.424797 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8efbf5-71ee-41aa-800d-98e3ad28289a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:37.425082 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8efbf5-71ee-41aa-800d-98e3ad28289a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:37.425361 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8efbf5-71ee-41aa-800d-98e3ad28289a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:37.433657 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8efbf5-71ee-41aa-800d-98e3ad28289a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:37.435118 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8efbf5-71ee-41aa-800d-98e3ad28289a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:37.449077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-6d8efbf5-71ee-41aa-800d-98e3ad28289a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 22:23:37.449077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:37.449077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:37.449077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:37.449077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:37.449077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:37.449077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:37.449077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:37.449077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:37.449077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:37.449077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:37.449077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:37.449077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:37.449077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:37.449077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:37.449077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:37.450255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:37.450255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:37.450255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:37.450255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:37.450255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:37.450255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:37.450255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:37.450255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:37.450255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:37.450255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:37.450255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:37.450255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:37.450255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:37.450255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:37.450255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:37.450255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:37.450255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:37.450255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:37.451260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:37.451260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:37.451260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:37.451260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:37.451260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:37.451260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:37.451260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:37.451260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:37.451260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 22:23:37.451260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:37.451260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 313/649] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:37.451860 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8efbf5-71ee-41aa-800d-98e3ad28289a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:37.468123 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:37.510677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b68816-89e1-4813-bf80-835c58d1e937 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:37.545639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbf7f3-700c-4339-acb8-6e0bc9739f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:37.546315 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbf7f3-700c-4339-acb8-6e0bc9739f6b None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:37.546605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbf7f3-700c-4339-acb8-6e0bc9739f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:37.546894 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbf7f3-700c-4339-acb8-6e0bc9739f6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:37.547754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbf7f3-700c-4339-acb8-6e0bc9739f6b None admin] RBAC: Adding query filter params (role.id=2a71a120d6e34544aa5a2031ddf65ec8, scope.project.id=56da0975019c44409b53be473131143a, user.id=35d686923ef541819de3c77847d38511) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:37.548076 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbf7f3-700c-4339-acb8-6e0bc9739f6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:37.549530 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbf7f3-700c-4339-acb8-6e0bc9739f6b None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:37.555940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 337/650] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:23:37 2022] GET /identity/v3/role_assignments?user.id=35d686923ef541819de3c77847d38511&scope.project.id=56da0975019c44409b53be473131143a&role.id=2a71a120d6e34544aa5a2031ddf65ec8 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:37.556586 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbf7f3-700c-4339-acb8-6e0bc9739f6b None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:37.570452 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8efbf5-71ee-41aa-800d-98e3ad28289a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:37.617748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa6b23-6d9e-4d39-894e-a89cabf3c64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:37.618983 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa6b23-6d9e-4d39-894e-a89cabf3c64e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:37.619979 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa6b23-6d9e-4d39-894e-a89cabf3c64e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:37.620416 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa6b23-6d9e-4d39-894e-a89cabf3c64e None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:37.621586 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa6b23-6d9e-4d39-894e-a89cabf3c64e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:37.622054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa6b23-6d9e-4d39-894e-a89cabf3c64e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:37.624448 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa6b23-6d9e-4d39-894e-a89cabf3c64e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:37.648515 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 314/651] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:37.648515 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa6b23-6d9e-4d39-894e-a89cabf3c64e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:37.682028 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:37.689170 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbf7f3-700c-4339-acb8-6e0bc9739f6b None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:37.719175 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:37.731784 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636e4bd-a5f5-40d2-9efe-ca44cfbbf10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:37.732748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636e4bd-a5f5-40d2-9efe-ca44cfbbf10b None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:37.733139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636e4bd-a5f5-40d2-9efe-ca44cfbbf10b None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:37.733520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636e4bd-a5f5-40d2-9efe-ca44cfbbf10b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:37.746429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636e4bd-a5f5-40d2-9efe-ca44cfbbf10b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:37.748167 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636e4bd-a5f5-40d2-9efe-ca44cfbbf10b None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:37.766379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-f636e4bd-a5f5-40d2-9efe-ca44cfbbf10b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:37.766379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:37.766379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:37.766379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:37.766379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:37.766379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:37.766379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:37.766379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:37.766379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:37.766379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:37.766379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:37.766379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:37.766379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:37.766379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:37.766379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:37.767509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:37.767509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:37.767509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:37.767509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:37.767509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:37.767509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:37.767509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:37.767509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:37.767509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:37.767509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:37.767509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:37.767509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:37.767509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:37.767509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:37.767509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:37.767509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:37.767509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:37.768591 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:37.768591 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:37.768591 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:37.768591 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:37.770072 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 338/652] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:37.770782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636e4bd-a5f5-40d2-9efe-ca44cfbbf10b None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:37.807453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa6b23-6d9e-4d39-894e-a89cabf3c64e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:37.859371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc123a0-aea3-4163-bc07-0b302ee979b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:37.860061 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc123a0-aea3-4163-bc07-0b302ee979b4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:37.860347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc123a0-aea3-4163-bc07-0b302ee979b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:37.860625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc123a0-aea3-4163-bc07-0b302ee979b4 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:37.861471 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc123a0-aea3-4163-bc07-0b302ee979b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:37.862212 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc123a0-aea3-4163-bc07-0b302ee979b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:37.864005 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc123a0-aea3-4163-bc07-0b302ee979b4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:37.885955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 315/653] 10.222.0.43 () {56 vars in 1137 bytes} [Thu Jun 23 22:23:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:37.886709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc123a0-aea3-4163-bc07-0b302ee979b4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:37.900133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636e4bd-a5f5-40d2-9efe-ca44cfbbf10b None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:37.943153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbdec0-bfd7-4217-a75d-3f4fd4d8e387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:37.944858 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbdec0-bfd7-4217-a75d-3f4fd4d8e387 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:37.945163 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbdec0-bfd7-4217-a75d-3f4fd4d8e387 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:37.945441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbdec0-bfd7-4217-a75d-3f4fd4d8e387 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:37.955103 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbdec0-bfd7-4217-a75d-3f4fd4d8e387 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:37.957278 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbdec0-bfd7-4217-a75d-3f4fd4d8e387 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:37.974005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-0bbbdec0-bfd7-4217-a75d-3f4fd4d8e387 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 22:23:37.974005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:37.974005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:37.974005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:37.974005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:37.974005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:37.974005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:37.974005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:37.974005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:37.974005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:37.974005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:37.974005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:37.974005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:37.974005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:37.974005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:37.974005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:37.976041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:37.976041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:37.976041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:37.976041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:37.976041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:37.976041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:37.976041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:37.976041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:37.976041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:37.976041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:37.976041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:37.976041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:37.976041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:37.976041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:37.976041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:37.976041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:37.976041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:37.976041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:37.978173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:37.978173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:37.978173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:37.978173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:37.978173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:37.978173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:37.978173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:37.978173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:37.978173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 22:23:37.978173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:37.978173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 339/654] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:37.978173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbdec0-bfd7-4217-a75d-3f4fd4d8e387 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:37.986946 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:38.014683 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc123a0-aea3-4163-bc07-0b302ee979b4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:38.026957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:38.063166 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad687a-db8d-44fd-9d7e-08cb17901362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:38.064007 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad687a-db8d-44fd-9d7e-08cb17901362 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:38.064299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad687a-db8d-44fd-9d7e-08cb17901362 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:38.064994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad687a-db8d-44fd-9d7e-08cb17901362 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:38.065738 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad687a-db8d-44fd-9d7e-08cb17901362 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:38.066051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad687a-db8d-44fd-9d7e-08cb17901362 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:38.067805 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad687a-db8d-44fd-9d7e-08cb17901362 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:38.084429 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 316/655] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:38.086968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad687a-db8d-44fd-9d7e-08cb17901362 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:38.097375 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbdec0-bfd7-4217-a75d-3f4fd4d8e387 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:38.153252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd1ba4-54e0-4ef2-808f-2485c74ba16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:38.153836 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd1ba4-54e0-4ef2-808f-2485c74ba16b None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:38.154738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd1ba4-54e0-4ef2-808f-2485c74ba16b None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:38.155068 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd1ba4-54e0-4ef2-808f-2485c74ba16b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:38.208168 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad687a-db8d-44fd-9d7e-08cb17901362 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:38.236178 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd1ba4-54e0-4ef2-808f-2485c74ba16b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:38.237672 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd1ba4-54e0-4ef2-808f-2485c74ba16b None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:38.248079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7109f2-059f-417f-8ba9-4d975b341eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:38.248849 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7109f2-059f-417f-8ba9-4d975b341eac None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:38.249247 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7109f2-059f-417f-8ba9-4d975b341eac None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:38.249540 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7109f2-059f-417f-8ba9-4d975b341eac None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:38.250387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7109f2-059f-417f-8ba9-4d975b341eac None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:38.250807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7109f2-059f-417f-8ba9-4d975b341eac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:38.252275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7109f2-059f-417f-8ba9-4d975b341eac None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:38.255140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:38.273078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 317/656] 10.222.0.43 () {56 vars in 1137 bytes} [Thu Jun 23 22:23:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:38.273925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7109f2-059f-417f-8ba9-4d975b341eac None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:38.281879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:38.351978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 340/657] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:38.353013 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd1ba4-54e0-4ef2-808f-2485c74ba16b None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:38.387175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7109f2-059f-417f-8ba9-4d975b341eac None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:38.426516 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b7978-b5ff-49e1-973f-cd93a075864c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:38.428356 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b7978-b5ff-49e1-973f-cd93a075864c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:38.429021 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b7978-b5ff-49e1-973f-cd93a075864c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:38.429568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b7978-b5ff-49e1-973f-cd93a075864c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:38.437775 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43b7978-b5ff-49e1-973f-cd93a075864c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:38.439672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43b7978-b5ff-49e1-973f-cd93a075864c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:38.456273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f43b7978-b5ff-49e1-973f-cd93a075864c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 22:23:38.456273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:38.456273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:38.456273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:38.456273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:38.456273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:38.456273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:38.456273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:38.456273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:38.456273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:38.456273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:38.456273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:38.456273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:38.456273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:38.456273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:38.456273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:38.457269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:38.457269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:38.457269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:38.457269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:38.457269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:38.457269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:38.457269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:38.457269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:38.457269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:38.457269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:38.457269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:38.457269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:38.457269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:38.457269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:38.457269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:38.457269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:38.457269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:38.457269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:38.458208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:38.458208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:38.458208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:38.458208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:38.458208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:38.458208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:38.458208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:38.458208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:38.458208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 22:23:38.458208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:38.460950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 318/658] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:38.461254 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd1ba4-54e0-4ef2-808f-2485c74ba16b None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:38.462161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b7978-b5ff-49e1-973f-cd93a075864c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:38.510178 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a5c35-30c7-4aa1-826f-3bca4eb51d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:38.510850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a5c35-30c7-4aa1-826f-3bca4eb51d71 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:38.511274 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a5c35-30c7-4aa1-826f-3bca4eb51d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:38.511450 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a5c35-30c7-4aa1-826f-3bca4eb51d71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:38.520336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:38.526572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:38.576607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43b7978-b5ff-49e1-973f-cd93a075864c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:38.587694 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a5c35-30c7-4aa1-826f-3bca4eb51d71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:38.589415 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a5c35-30c7-4aa1-826f-3bca4eb51d71 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:38.614453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e717703-4e7f-400c-b860-954622c60ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:38.616260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e717703-4e7f-400c-b860-954622c60ccf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:38.616780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e717703-4e7f-400c-b860-954622c60ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:38.617261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e717703-4e7f-400c-b860-954622c60ccf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:38.628660 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e717703-4e7f-400c-b860-954622c60ccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:38.630473 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e717703-4e7f-400c-b860-954622c60ccf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:38.651110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4e717703-4e7f-400c-b860-954622c60ccf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:38.651110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:38.651110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:38.651110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:38.651110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:38.651110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:38.651110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:38.651110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:38.651110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:38.651110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:38.651110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:38.651110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:38.651110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:38.651110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:38.651110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:38.652058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:38.652058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:38.652058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:38.652058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:38.652058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:38.652058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:38.652058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:38.652058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:38.652058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:38.652058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:38.652058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:38.652058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:38.652058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:38.652058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:38.652058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:38.652058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:38.652058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:38.653544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:38.653544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:38.653544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:38.653544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:38.653544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 319/659] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:38.653544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e717703-4e7f-400c-b860-954622c60ccf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:38.706557 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 341/660] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:38.707729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a5c35-30c7-4aa1-826f-3bca4eb51d71 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:38.749350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:38.782326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e717703-4e7f-400c-b860-954622c60ccf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:38.791437 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:38.826635 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8d4fa-8be9-4560-92ae-dd222da31c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:38.827423 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8d4fa-8be9-4560-92ae-dd222da31c3c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:38.828143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8d4fa-8be9-4560-92ae-dd222da31c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:38.828367 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8d4fa-8be9-4560-92ae-dd222da31c3c None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:38.829234 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8d4fa-8be9-4560-92ae-dd222da31c3c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:38.829633 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8d4fa-8be9-4560-92ae-dd222da31c3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:38.831309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8d4fa-8be9-4560-92ae-dd222da31c3c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:38.835922 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a5c35-30c7-4aa1-826f-3bca4eb51d71 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:38.854767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 320/661] 10.222.0.43 () {56 vars in 1139 bytes} [Thu Jun 23 22:23:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:38.855933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8d4fa-8be9-4560-92ae-dd222da31c3c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:38.883756 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202ac5b-0304-4697-8169-33a900d0b860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:38.885221 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202ac5b-0304-4697-8169-33a900d0b860 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:38.885799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202ac5b-0304-4697-8169-33a900d0b860 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:38.886240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202ac5b-0304-4697-8169-33a900d0b860 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:38.887305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202ac5b-0304-4697-8169-33a900d0b860 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:38.888074 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202ac5b-0304-4697-8169-33a900d0b860 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:38.889894 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202ac5b-0304-4697-8169-33a900d0b860 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:38.906672 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 342/662] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:38.908241 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202ac5b-0304-4697-8169-33a900d0b860 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:38.979981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8d4fa-8be9-4560-92ae-dd222da31c3c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:39.020999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dabd63-b1c3-4fe8-bb1a-f4a737d50438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:39.021839 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dabd63-b1c3-4fe8-bb1a-f4a737d50438 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:39.022155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dabd63-b1c3-4fe8-bb1a-f4a737d50438 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:39.022671 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dabd63-b1c3-4fe8-bb1a-f4a737d50438 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:39.033089 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:39.033215 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dabd63-b1c3-4fe8-bb1a-f4a737d50438 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:39.034668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dabd63-b1c3-4fe8-bb1a-f4a737d50438 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:39.037727 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202ac5b-0304-4697-8169-33a900d0b860 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:39.054413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e1dabd63-b1c3-4fe8-bb1a-f4a737d50438 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:39.054413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:39.054413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:39.054413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:39.054413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:39.054413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:39.054413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:39.054413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:39.054413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:39.054413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:39.054413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:39.054413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:39.054413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:39.054413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:39.054413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:39.055445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:39.055445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:39.055445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:39.055445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:39.055445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:39.055445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:39.055445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:39.055445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:39.055445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:39.055445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:39.055445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:39.055445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:39.055445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:39.055445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:39.055445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:39.055445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:39.055445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:39.056384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:39.056384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:39.056384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:39.056384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:39.056384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 321/663] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:39.056656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dabd63-b1c3-4fe8-bb1a-f4a737d50438 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:39.080016 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:39.080256 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857634d-4fce-40d2-a52b-eb52f5391e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:39.081116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857634d-4fce-40d2-a52b-eb52f5391e18 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:39.081221 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857634d-4fce-40d2-a52b-eb52f5391e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:39.081502 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857634d-4fce-40d2-a52b-eb52f5391e18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:39.158188 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857634d-4fce-40d2-a52b-eb52f5391e18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:39.159904 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857634d-4fce-40d2-a52b-eb52f5391e18 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:39.176021 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dabd63-b1c3-4fe8-bb1a-f4a737d50438 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:39.209821 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5013a5-9cb0-4405-8595-f2d636f7cdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:39.210889 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5013a5-9cb0-4405-8595-f2d636f7cdcb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:39.211272 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5013a5-9cb0-4405-8595-f2d636f7cdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:39.211620 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5013a5-9cb0-4405-8595-f2d636f7cdcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:39.218646 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5013a5-9cb0-4405-8595-f2d636f7cdcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:39.220204 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5013a5-9cb0-4405-8595-f2d636f7cdcb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:39.235020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-aa5013a5-9cb0-4405-8595-f2d636f7cdcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:39.235020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:39.235020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:39.235020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:39.235020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:39.235020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:39.235020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:39.235020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:39.235020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:39.235020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:39.235020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:39.235020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:39.235020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:39.235020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:39.235020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:39.235966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:39.235966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:39.235966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:39.235966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:39.235966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:39.235966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:39.235966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:39.235966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:39.235966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:39.235966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:39.235966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:39.235966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:39.235966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:39.235966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:39.235966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:39.235966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:39.235966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:39.236885 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:39.236885 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:39.236885 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:39.236885 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:39.236885 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:39.236885 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:39.236885 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:39.236885 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:39.241238 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 322/664] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:39.242295 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebad8f9-2e6d-401d-a435-d05fd904fab0 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:39.242592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebad8f9-2e6d-401d-a435-d05fd904fab0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:39.242878 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebad8f9-2e6d-401d-a435-d05fd904fab0 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:39.247075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 323/665] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:39.247075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebad8f9-2e6d-401d-a435-d05fd904fab0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:39.269068 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:39.280019 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 343/666] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:39.280019 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857634d-4fce-40d2-a52b-eb52f5391e18 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:39.333168 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:39.361317 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebad8f9-2e6d-401d-a435-d05fd904fab0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:39.391933 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857634d-4fce-40d2-a52b-eb52f5391e18 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:39.401978 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3ca93-f37b-4598-a0bd-fca4c1f68f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:39.402715 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3ca93-f37b-4598-a0bd-fca4c1f68f3b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:39.403355 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3ca93-f37b-4598-a0bd-fca4c1f68f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:39.403696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3ca93-f37b-4598-a0bd-fca4c1f68f3b None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:39.404622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3ca93-f37b-4598-a0bd-fca4c1f68f3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:39.404948 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3ca93-f37b-4598-a0bd-fca4c1f68f3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:39.406331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3ca93-f37b-4598-a0bd-fca4c1f68f3b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:39.418344 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 324/667] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:39.420423 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb89ac0-ce4c-4a63-83ae-1775dd211180 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:39.420718 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb89ac0-ce4c-4a63-83ae-1775dd211180 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:39.421001 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb89ac0-ce4c-4a63-83ae-1775dd211180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:39.436275 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f3d29-5e79-49d9-b55f-01d968fb1a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:39.437859 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f3d29-5e79-49d9-b55f-01d968fb1a2c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:39.438645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f3d29-5e79-49d9-b55f-01d968fb1a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:39.439407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f3d29-5e79-49d9-b55f-01d968fb1a2c None admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:39.440024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8fb89ac0-ce4c-4a63-83ae-1775dd211180 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:39.442228 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8fb89ac0-ce4c-4a63-83ae-1775dd211180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:39.442741 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f3d29-5e79-49d9-b55f-01d968fb1a2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:39.443162 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f3d29-5e79-49d9-b55f-01d968fb1a2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:39.447788 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f3d29-5e79-49d9-b55f-01d968fb1a2c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:39.460686 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 344/668] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:39.461519 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f3d29-5e79-49d9-b55f-01d968fb1a2c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:39.520814 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 325/669] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:39.521640 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb89ac0-ce4c-4a63-83ae-1775dd211180 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:39.530434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:39.579741 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f3d29-5e79-49d9-b55f-01d968fb1a2c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:39.594715 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:39.620360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d807db7-dfff-4d7f-8407-9b4e781a6042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:39.621174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d807db7-dfff-4d7f-8407-9b4e781a6042 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:39.621439 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d807db7-dfff-4d7f-8407-9b4e781a6042 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:39.621719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d807db7-dfff-4d7f-8407-9b4e781a6042 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:39.632480 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb89ac0-ce4c-4a63-83ae-1775dd211180 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:39.633011 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d807db7-dfff-4d7f-8407-9b4e781a6042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:39.634452 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d807db7-dfff-4d7f-8407-9b4e781a6042 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:39.652070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-1d807db7-dfff-4d7f-8407-9b4e781a6042 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:39.652070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:39.652070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:39.652070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:39.652070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:39.652070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:39.652070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:39.652070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:39.652070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:39.652070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:39.652070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:39.652070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:39.652070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:39.652070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:39.652070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:39.653015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:39.653015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:39.653015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:39.653015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:39.653015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:39.653015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:39.653015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:39.653015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:39.653015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:39.653015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:39.653015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:39.653015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:39.653015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:39.653015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:39.653015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:39.653015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:39.653015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:39.653932 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:39.653932 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:39.653932 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:39.653932 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:39.653932 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 345/670] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:39.654723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d807db7-dfff-4d7f-8407-9b4e781a6042 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:39.671978 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca317b0-ffb7-4356-a5ff-9efd39075cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:39.672770 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca317b0-ffb7-4356-a5ff-9efd39075cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:39.673052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca317b0-ffb7-4356-a5ff-9efd39075cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:39.673331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca317b0-ffb7-4356-a5ff-9efd39075cd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:39.681420 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca317b0-ffb7-4356-a5ff-9efd39075cd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:39.682861 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca317b0-ffb7-4356-a5ff-9efd39075cd0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:39.696955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-aca317b0-ffb7-4356-a5ff-9efd39075cd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:39.696955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:39.696955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:39.696955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:39.696955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:39.696955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:39.696955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:39.696955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:39.696955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:39.696955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:39.696955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:39.696955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:39.696955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:39.696955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:39.696955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:39.697869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:39.697869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:39.697869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:39.697869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:39.697869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:39.697869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:39.697869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:39.697869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:39.697869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:39.697869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:39.697869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:39.697869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:39.697869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:39.697869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:39.697869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:39.697869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:39.697869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:39.698761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:39.698761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:39.698761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:39.698761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:39.698761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:39.698761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:39.698761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:39.698761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:39.698761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 326/671] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:39.699842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca317b0-ffb7-4356-a5ff-9efd39075cd0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:39.764925 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d807db7-dfff-4d7f-8407-9b4e781a6042 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:39.775704 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:39.808559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941781c1-f070-439e-bc3e-a611716ded4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:39.809337 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941781c1-f070-439e-bc3e-a611716ded4c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:39.809687 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941781c1-f070-439e-bc3e-a611716ded4c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:39.809993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941781c1-f070-439e-bc3e-a611716ded4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:39.810867 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941781c1-f070-439e-bc3e-a611716ded4c None admin] RBAC: Adding query filter params (role.id=2a71a120d6e34544aa5a2031ddf65ec8, scope.project.id=56da0975019c44409b53be473131143a, user.id=36de21e0b3de48dabe0f27a9db8a37aa) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:39.811224 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941781c1-f070-439e-bc3e-a611716ded4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:39.812690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941781c1-f070-439e-bc3e-a611716ded4c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:39.814153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca317b0-ffb7-4356-a5ff-9efd39075cd0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:39.826944 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 346/672] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:23:39 2022] GET /identity/v3/role_assignments?user.id=36de21e0b3de48dabe0f27a9db8a37aa&scope.project.id=56da0975019c44409b53be473131143a&role.id=2a71a120d6e34544aa5a2031ddf65ec8 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:39.826944 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941781c1-f070-439e-bc3e-a611716ded4c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:39.853058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:39.864028 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adeb48d-1c97-4089-ad27-a96c2559b534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:39.865143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adeb48d-1c97-4089-ad27-a96c2559b534 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:39.865614 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adeb48d-1c97-4089-ad27-a96c2559b534 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:39.865894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adeb48d-1c97-4089-ad27-a96c2559b534 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:39.878507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adeb48d-1c97-4089-ad27-a96c2559b534 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:39.887003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1adeb48d-1c97-4089-ad27-a96c2559b534 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:39.887003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:39.887003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:39.887003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:39.887003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:39.887003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:39.887003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:39.887003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:39.887003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:39.887003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:39.887003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:39.887003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:39.887003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:39.887003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:39.887003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:39.887003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:39.888018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:39.888018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:39.888018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:39.888018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:39.888018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:39.888018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:39.888018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:39.888018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:39.888018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:39.888018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:39.888018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:39.888018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:39.888018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:39.888018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:39.888018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:39.888018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:39.888856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:39.888856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:39.888856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:39.888856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:39.888856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:39.888856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:39.888856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:39.888856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:39.888856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:39.888856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:39.888856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:39.888856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:39.888856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 327/673] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:39.889555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adeb48d-1c97-4089-ad27-a96c2559b534 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:39.950223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941781c1-f070-439e-bc3e-a611716ded4c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:39.994224 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35517100-84b6-4f1a-820f-8eedf0f54f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:39.995300 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35517100-84b6-4f1a-820f-8eedf0f54f11 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:39.995745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35517100-84b6-4f1a-820f-8eedf0f54f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:39.996482 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35517100-84b6-4f1a-820f-8eedf0f54f11 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:39.997479 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35517100-84b6-4f1a-820f-8eedf0f54f11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:39.997800 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35517100-84b6-4f1a-820f-8eedf0f54f11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:39.999674 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35517100-84b6-4f1a-820f-8eedf0f54f11 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:40.010798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adeb48d-1c97-4089-ad27-a96c2559b534 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:40.012354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 347/674] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:40.013498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76a7bc-666c-4e10-8a62-523209c5abb2 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:40.013856 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76a7bc-666c-4e10-8a62-523209c5abb2 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:40.014146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76a7bc-666c-4e10-8a62-523209c5abb2 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:40.015640 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 348/675] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:40.016395 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76a7bc-666c-4e10-8a62-523209c5abb2 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:40.055629 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea49011-6f5a-4a97-87cd-96e6c99529ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:40.056221 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea49011-6f5a-4a97-87cd-96e6c99529ef None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:40.058580 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea49011-6f5a-4a97-87cd-96e6c99529ef None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:40.058580 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea49011-6f5a-4a97-87cd-96e6c99529ef None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:40.058580 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea49011-6f5a-4a97-87cd-96e6c99529ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:40.058580 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea49011-6f5a-4a97-87cd-96e6c99529ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:40.059336 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea49011-6f5a-4a97-87cd-96e6c99529ef None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:40.060319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:40.076964 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 328/676] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:40.076964 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea49011-6f5a-4a97-87cd-96e6c99529ef None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:40.124264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:40.149632 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76a7bc-666c-4e10-8a62-523209c5abb2 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:40.182298 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea49011-6f5a-4a97-87cd-96e6c99529ef None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:40.190404 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ec470-f64a-4049-a1b7-bf60c67be945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:40.191116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ec470-f64a-4049-a1b7-bf60c67be945 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:40.191402 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ec470-f64a-4049-a1b7-bf60c67be945 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:40.191683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ec470-f64a-4049-a1b7-bf60c67be945 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:40.192435 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ec470-f64a-4049-a1b7-bf60c67be945 None admin] RBAC: Adding query filter params (name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:40.192747 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ec470-f64a-4049-a1b7-bf60c67be945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:40.194227 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ec470-f64a-4049-a1b7-bf60c67be945 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:40.202812 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 349/677] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:23:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:40.204320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da129a6-bf98-4ed5-8b09-c9819133d013 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:40.204828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da129a6-bf98-4ed5-8b09-c9819133d013 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:40.205302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da129a6-bf98-4ed5-8b09-c9819133d013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:40.221742 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-1da129a6-bf98-4ed5-8b09-c9819133d013 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:40.222799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6153d-3bb0-4620-9afa-fdd02401df4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:40.223030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1da129a6-bf98-4ed5-8b09-c9819133d013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:40.223633 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6153d-3bb0-4620-9afa-fdd02401df4f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:40.223923 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6153d-3bb0-4620-9afa-fdd02401df4f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:40.224203 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6153d-3bb0-4620-9afa-fdd02401df4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:40.233174 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a6153d-3bb0-4620-9afa-fdd02401df4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:40.234616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a6153d-3bb0-4620-9afa-fdd02401df4f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:40.248909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-15a6153d-3bb0-4620-9afa-fdd02401df4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:40.248909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:40.248909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:40.248909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:40.248909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:40.248909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:40.248909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:40.248909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:40.248909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:40.248909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:40.248909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:40.248909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:40.248909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:40.248909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:40.248909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:40.249823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:40.249823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:40.249823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:40.249823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:40.249823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:40.249823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:40.249823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:40.249823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:40.249823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:40.249823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:40.249823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:40.249823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:40.249823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:40.249823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:40.249823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:40.249823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:40.249823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:40.250709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:40.250709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:40.250709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:40.250709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:40.250709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:40.250709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:40.250709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:40.250709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:40.250709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 329/678] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:40.251218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6153d-3bb0-4620-9afa-fdd02401df4f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:40.306119 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 350/679] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:40.306899 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da129a6-bf98-4ed5-8b09-c9819133d013 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:40.344815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:40.381549 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6153d-3bb0-4620-9afa-fdd02401df4f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:40.389374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:40.430659 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7854966-8abd-4ab7-93d0-22a41be3eb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:40.431420 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7854966-8abd-4ab7-93d0-22a41be3eb11 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:40.431683 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7854966-8abd-4ab7-93d0-22a41be3eb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:40.431965 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7854966-8abd-4ab7-93d0-22a41be3eb11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:40.432807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7854966-8abd-4ab7-93d0-22a41be3eb11 None admin] RBAC: Adding query filter params (role.id=2a71a120d6e34544aa5a2031ddf65ec8, scope.project.id=56da0975019c44409b53be473131143a, user.id=95bc067d93fa4207bec1c85600728085) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:40.433128 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7854966-8abd-4ab7-93d0-22a41be3eb11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:40.434554 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7854966-8abd-4ab7-93d0-22a41be3eb11 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:40.450964 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 330/680] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:23:40 2022] GET /identity/v3/role_assignments?user.id=95bc067d93fa4207bec1c85600728085&scope.project.id=56da0975019c44409b53be473131143a&role.id=2a71a120d6e34544aa5a2031ddf65ec8 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:40.450964 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7854966-8abd-4ab7-93d0-22a41be3eb11 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:40.454267 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da129a6-bf98-4ed5-8b09-c9819133d013 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:40.500381 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff6d99-40a3-4b9c-b256-fb726ab55682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:40.501639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff6d99-40a3-4b9c-b256-fb726ab55682 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:40.501929 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff6d99-40a3-4b9c-b256-fb726ab55682 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:40.502678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff6d99-40a3-4b9c-b256-fb726ab55682 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:40.517348 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff6d99-40a3-4b9c-b256-fb726ab55682 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:40.519655 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff6d99-40a3-4b9c-b256-fb726ab55682 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:40.551554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-e1ff6d99-40a3-4b9c-b256-fb726ab55682 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:40.551554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:40.551554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:40.551554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:40.551554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:40.551554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:40.551554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:40.551554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:40.551554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:40.551554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:40.551554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:40.551554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:40.551554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:40.551554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:40.551554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:40.552526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:40.552526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:40.552526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:40.552526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:40.552526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:40.552526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:40.552526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:40.552526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:40.552526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:40.552526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:40.552526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:40.552526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:40.552526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:40.552526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:40.552526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:40.552526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:40.552526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:40.554149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:40.554149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:40.554149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:40.554149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:40.556569 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 351/681] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:40.557235 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff6d99-40a3-4b9c-b256-fb726ab55682 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:40.585635 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7854966-8abd-4ab7-93d0-22a41be3eb11 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:40.625730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ef44f-3efc-4de1-86fc-c4f82308849a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:40.626408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ef44f-3efc-4de1-86fc-c4f82308849a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:40.626773 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ef44f-3efc-4de1-86fc-c4f82308849a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:40.627090 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ef44f-3efc-4de1-86fc-c4f82308849a None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:40.627941 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ef44f-3efc-4de1-86fc-c4f82308849a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:40.628259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ef44f-3efc-4de1-86fc-c4f82308849a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:40.629752 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ef44f-3efc-4de1-86fc-c4f82308849a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:40.649473 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 331/682] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:40.649473 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ef44f-3efc-4de1-86fc-c4f82308849a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:40.682740 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:40.688236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:40.717388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff6d99-40a3-4b9c-b256-fb726ab55682 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:40.768504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cf933-8cfd-40f6-9538-49c8057f1a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:40.769304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cf933-8cfd-40f6-9538-49c8057f1a28 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:40.769773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cf933-8cfd-40f6-9538-49c8057f1a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:40.770885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cf933-8cfd-40f6-9538-49c8057f1a28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:40.792311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cf933-8cfd-40f6-9538-49c8057f1a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:40.797855 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ef44f-3efc-4de1-86fc-c4f82308849a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:40.800770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-437cf933-8cfd-40f6-9538-49c8057f1a28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:23:40.800770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:40.800770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:40.800770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:40.800770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:40.800770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:40.800770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:40.800770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:40.800770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:40.800770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:40.800770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:40.800770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:40.800770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:40.800770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:40.800770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:40.800770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:40.805252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:40.805252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:40.805252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:40.805252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:40.805252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:40.805252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:40.805252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:40.805252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:40.805252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:40.805252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:40.805252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:40.805252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:40.805252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:40.805252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:40.805252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:40.805252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:40.807350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:40.807350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:40.807350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:40.807350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:40.807350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:40.807350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:40.807350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:40.807350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:40.807350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:40.807350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:40.807350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:23:40.807350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:40.807350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 352/683] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:23:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:40.809274 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cf933-8cfd-40f6-9538-49c8057f1a28 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:40.856195 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e88d1-d5dc-4d73-8cbc-11de5ae6323e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:40.857727 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e88d1-d5dc-4d73-8cbc-11de5ae6323e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:40.858626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e88d1-d5dc-4d73-8cbc-11de5ae6323e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:40.860356 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e88d1-d5dc-4d73-8cbc-11de5ae6323e None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:40.861155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e88d1-d5dc-4d73-8cbc-11de5ae6323e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:40.861474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e88d1-d5dc-4d73-8cbc-11de5ae6323e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:40.862872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e88d1-d5dc-4d73-8cbc-11de5ae6323e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:40.881901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 332/684] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:40.881901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e88d1-d5dc-4d73-8cbc-11de5ae6323e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:40.956648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cf933-8cfd-40f6-9538-49c8057f1a28 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:40.994431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:41.004942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a862819-cdcb-4102-a45e-10b665a32fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:41.006457 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a862819-cdcb-4102-a45e-10b665a32fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:41.007073 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a862819-cdcb-4102-a45e-10b665a32fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:41.007217 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e88d1-d5dc-4d73-8cbc-11de5ae6323e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:41.007683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a862819-cdcb-4102-a45e-10b665a32fc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:41.008667 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a862819-cdcb-4102-a45e-10b665a32fc1 None admin] RBAC: Adding query filter params (role.id=2a71a120d6e34544aa5a2031ddf65ec8, scope.project.id=56da0975019c44409b53be473131143a, user.id=37ba527b96634fc2a48b6ca9597d8191) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:41.009314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a862819-cdcb-4102-a45e-10b665a32fc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:41.012327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a862819-cdcb-4102-a45e-10b665a32fc1 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:41.022894 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 353/685] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:23:40 2022] GET /identity/v3/role_assignments?user.id=37ba527b96634fc2a48b6ca9597d8191&scope.project.id=56da0975019c44409b53be473131143a&role.id=2a71a120d6e34544aa5a2031ddf65ec8 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:41.024003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a862819-cdcb-4102-a45e-10b665a32fc1 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:41.027191 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:41.109078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba2aee-1ca0-42ee-a540-424d2fd0290b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:41.109692 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba2aee-1ca0-42ee-a540-424d2fd0290b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:41.109990 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba2aee-1ca0-42ee-a540-424d2fd0290b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:41.110281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aba2aee-1ca0-42ee-a540-424d2fd0290b None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:41.111209 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba2aee-1ca0-42ee-a540-424d2fd0290b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:41.111527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba2aee-1ca0-42ee-a540-424d2fd0290b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:41.112971 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aba2aee-1ca0-42ee-a540-424d2fd0290b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:41.145499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 333/686] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:41.152114 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba2aee-1ca0-42ee-a540-424d2fd0290b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:41.165169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a862819-cdcb-4102-a45e-10b665a32fc1 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:41.207757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056133d-e315-43d5-8881-832faf34a491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:41.208586 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056133d-e315-43d5-8881-832faf34a491 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:41.208882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056133d-e315-43d5-8881-832faf34a491 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:41.209366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056133d-e315-43d5-8881-832faf34a491 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:41.221350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056133d-e315-43d5-8881-832faf34a491 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:41.223651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056133d-e315-43d5-8881-832faf34a491 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:41.241759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-c056133d-e315-43d5-8881-832faf34a491 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:23:41.241759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:41.241759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:41.241759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:41.241759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:41.241759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:41.241759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:41.241759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:41.241759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:41.241759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:41.241759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:41.241759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:41.241759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:41.241759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:41.241759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:41.241759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:41.242888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:41.242888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:41.242888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:41.242888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:41.242888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:41.242888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:41.242888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:41.242888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:41.242888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:41.242888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:41.242888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:41.242888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:41.242888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:41.242888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:41.242888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:41.242888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:41.242888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:41.242888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:41.244082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:41.244082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:41.244082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:41.244082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:41.244082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:41.244082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:41.244082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:41.244082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:41.244082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:23:41.244082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:41.244082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 354/687] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:23:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:41.249816 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056133d-e315-43d5-8881-832faf34a491 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:41.346810 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:41.368244 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aba2aee-1ca0-42ee-a540-424d2fd0290b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:41.411429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056133d-e315-43d5-8881-832faf34a491 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:41.421701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d7423-d80f-4e05-9645-b17816c619e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:41.422569 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d7423-d80f-4e05-9645-b17816c619e2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:41.422969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d7423-d80f-4e05-9645-b17816c619e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:41.423262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d7423-d80f-4e05-9645-b17816c619e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:41.442242 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d7423-d80f-4e05-9645-b17816c619e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:41.444532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d7423-d80f-4e05-9645-b17816c619e2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:41.457187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166443d-da72-4630-a454-eac60c23c06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:41.458025 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166443d-da72-4630-a454-eac60c23c06c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:41.458411 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166443d-da72-4630-a454-eac60c23c06c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:41.458701 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166443d-da72-4630-a454-eac60c23c06c None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:41.460132 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166443d-da72-4630-a454-eac60c23c06c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:41.460643 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166443d-da72-4630-a454-eac60c23c06c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:41.462736 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f166443d-da72-4630-a454-eac60c23c06c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:41.470309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-018d7423-d80f-4e05-9645-b17816c619e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:41.470309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:41.470309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:41.470309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:41.470309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:41.470309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:41.470309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:41.470309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:41.470309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:41.470309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:41.470309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:41.470309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:41.470309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:41.470309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:41.470309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:41.471271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:41.471271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:41.471271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:41.471271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:41.471271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:41.471271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:41.471271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:41.471271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:41.471271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:41.471271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:41.471271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:41.471271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:41.471271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:41.471271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:41.471271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:41.471271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:41.471271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:41.472179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:41.472179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:41.472179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:41.472179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:41.472179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 334/688] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:41.480436 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d7423-d80f-4e05-9645-b17816c619e2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:41.485288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:41.502514 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 355/689] 10.222.0.43 () {56 vars in 1135 bytes} [Thu Jun 23 22:23:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:41.513134 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166443d-da72-4630-a454-eac60c23c06c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:41.680164 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166443d-da72-4630-a454-eac60c23c06c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:41.683674 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d7423-d80f-4e05-9645-b17816c619e2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:41.703308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:41.723472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17850e83-4fa1-40ef-b312-f761d58b03dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:41.724494 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17850e83-4fa1-40ef-b312-f761d58b03dc None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:41.727249 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17850e83-4fa1-40ef-b312-f761d58b03dc None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:41.727749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17850e83-4fa1-40ef-b312-f761d58b03dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:41.754331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e409da9-b7dd-42ad-afeb-0c069006b4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:41.757055 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e409da9-b7dd-42ad-afeb-0c069006b4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:41.757674 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e409da9-b7dd-42ad-afeb-0c069006b4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:41.759375 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e409da9-b7dd-42ad-afeb-0c069006b4d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:41.760326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e409da9-b7dd-42ad-afeb-0c069006b4d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:41.762447 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e409da9-b7dd-42ad-afeb-0c069006b4d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:41.764348 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e409da9-b7dd-42ad-afeb-0c069006b4d2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:41.780356 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 335/690] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:41.786182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e409da9-b7dd-42ad-afeb-0c069006b4d2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:41.871792 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17850e83-4fa1-40ef-b312-f761d58b03dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:41.873250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17850e83-4fa1-40ef-b312-f761d58b03dc None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:41.891239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:41.971470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e409da9-b7dd-42ad-afeb-0c069006b4d2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:42.010159 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce9699-d265-474a-ab76-28492b5b7193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:42.011415 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce9699-d265-474a-ab76-28492b5b7193 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:42.011939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce9699-d265-474a-ab76-28492b5b7193 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:42.012432 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce9699-d265-474a-ab76-28492b5b7193 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:42.020665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce9699-d265-474a-ab76-28492b5b7193 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:42.023216 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce9699-d265-474a-ab76-28492b5b7193 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:42.038702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-c2ce9699-d265-474a-ab76-28492b5b7193 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:23:42.038702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:42.038702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:42.038702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:42.038702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:42.038702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:42.038702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:42.038702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:42.038702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:42.038702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:42.038702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:42.038702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:42.038702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:42.038702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:42.038702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:42.038702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:42.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:42.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:42.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:42.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:42.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:42.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:42.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:42.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:42.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:42.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:42.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:42.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:42.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:42.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:42.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:42.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:42.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:42.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:42.040746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:42.040746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:42.040746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:42.040746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:42.040746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:42.040746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:42.040746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:42.040746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:42.040746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:23:42.040746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:42.043358 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 336/691] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Jun 23 22:23:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:42.049079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce9699-d265-474a-ab76-28492b5b7193 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:42.055603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 356/692] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:42.060953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17850e83-4fa1-40ef-b312-f761d58b03dc None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:42.101976 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:42.189321 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17850e83-4fa1-40ef-b312-f761d58b03dc None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:42.224299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce9699-d265-474a-ab76-28492b5b7193 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:42.227228 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004091e6-b443-4710-ace9-38fc0270f500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:42.228042 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004091e6-b443-4710-ace9-38fc0270f500 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:42.228335 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004091e6-b443-4710-ace9-38fc0270f500 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:42.228621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004091e6-b443-4710-ace9-38fc0270f500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:42.239112 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004091e6-b443-4710-ace9-38fc0270f500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:42.240656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004091e6-b443-4710-ace9-38fc0270f500 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:42.250595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:42.258767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-004091e6-b443-4710-ace9-38fc0270f500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:42.258767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:42.258767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:42.258767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:42.258767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:42.258767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:42.258767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:42.258767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:42.258767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:42.258767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:42.258767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:42.258767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:42.258767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:42.258767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:42.258767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:42.259864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:42.259864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:42.259864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:42.259864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:42.259864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:42.259864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:42.259864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:42.259864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:42.259864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:42.259864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:42.259864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:42.259864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:42.259864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:42.259864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:42.259864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:42.259864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:42.259864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:42.261101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:42.261101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:42.261101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:42.261101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:42.261101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 357/693] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:23:42.261713 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f269e-09fc-4cd2-a96f-0d35b7defcc8 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:42.262050 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f269e-09fc-4cd2-a96f-0d35b7defcc8 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:42.262257 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f269e-09fc-4cd2-a96f-0d35b7defcc8 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:42.267725 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 358/694] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:42.268339 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f269e-09fc-4cd2-a96f-0d35b7defcc8 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:42.268743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf86649-d2b0-4046-b117-0a7254466e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:42.269751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf86649-d2b0-4046-b117-0a7254466e03 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:42.270231 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf86649-d2b0-4046-b117-0a7254466e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:42.270688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf86649-d2b0-4046-b117-0a7254466e03 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:42.273297 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf86649-d2b0-4046-b117-0a7254466e03 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:42.273803 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf86649-d2b0-4046-b117-0a7254466e03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:42.275465 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf86649-d2b0-4046-b117-0a7254466e03 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:42.293855 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 337/695] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Jun 23 22:23:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:42.295220 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784fac4-3e49-4eb0-90aa-b9f08fab8077 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:42.295526 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784fac4-3e49-4eb0-90aa-b9f08fab8077 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:42.295795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784fac4-3e49-4eb0-90aa-b9f08fab8077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:42.315648 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a784fac4-3e49-4eb0-90aa-b9f08fab8077 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:42.316851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a784fac4-3e49-4eb0-90aa-b9f08fab8077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:42.347476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:42.382947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f269e-09fc-4cd2-a96f-0d35b7defcc8 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:42.406952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 338/696] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:42.406952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784fac4-3e49-4eb0-90aa-b9f08fab8077 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:42.452265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4d2eb-c457-4e31-bb07-42990644c045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:42.452984 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4d2eb-c457-4e31-bb07-42990644c045 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:42.453275 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4d2eb-c457-4e31-bb07-42990644c045 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:42.453563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4d2eb-c457-4e31-bb07-42990644c045 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:42.454494 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4d2eb-c457-4e31-bb07-42990644c045 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:42.454819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4d2eb-c457-4e31-bb07-42990644c045 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:42.456956 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4d2eb-c457-4e31-bb07-42990644c045 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:42.472932 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 359/697] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:42.473843 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4d2eb-c457-4e31-bb07-42990644c045 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:42.510535 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784fac4-3e49-4eb0-90aa-b9f08fab8077 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:42.527440 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:42.548146 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6ae6b-5a47-4476-89b6-db430f52acf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:42.549454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6ae6b-5a47-4476-89b6-db430f52acf6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:42.550126 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6ae6b-5a47-4476-89b6-db430f52acf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:42.550555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6ae6b-5a47-4476-89b6-db430f52acf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:42.604074 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4d2eb-c457-4e31-bb07-42990644c045 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:42.638269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6ae6b-5a47-4476-89b6-db430f52acf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:42.640730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6ae6b-5a47-4476-89b6-db430f52acf6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:42.647890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:42.652266 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a92f48-aea4-4b00-bf65-2ce8c63c2f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:42.653126 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a92f48-aea4-4b00-bf65-2ce8c63c2f10 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:42.653416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a92f48-aea4-4b00-bf65-2ce8c63c2f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:42.653697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a92f48-aea4-4b00-bf65-2ce8c63c2f10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:42.666049 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a92f48-aea4-4b00-bf65-2ce8c63c2f10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:42.667553 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a92f48-aea4-4b00-bf65-2ce8c63c2f10 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:42.685486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-f2a92f48-aea4-4b00-bf65-2ce8c63c2f10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:42.685486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:42.685486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:42.685486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:42.685486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:42.685486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:42.685486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:42.685486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:42.685486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:42.685486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:42.685486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:42.685486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:42.685486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:42.685486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:42.685486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:42.686578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:42.686578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:42.686578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:42.686578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:42.686578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:42.686578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:42.686578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:42.686578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:42.686578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:42.686578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:42.686578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:42.686578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:42.686578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:42.686578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:42.686578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:42.686578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:42.686578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:42.687543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:42.687543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:42.687543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:42.687543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:42.691483 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 360/698] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:42.692624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a92f48-aea4-4b00-bf65-2ce8c63c2f10 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:42.767365 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 339/699] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:42.768568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6ae6b-5a47-4476-89b6-db430f52acf6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:42.786372 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:42.800317 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a92f48-aea4-4b00-bf65-2ce8c63c2f10 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:42.840751 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45faaa-69de-48eb-98d0-d6c2ea317883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:42.841544 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45faaa-69de-48eb-98d0-d6c2ea317883 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:42.841832 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45faaa-69de-48eb-98d0-d6c2ea317883 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:42.842109 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45faaa-69de-48eb-98d0-d6c2ea317883 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:42.849864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45faaa-69de-48eb-98d0-d6c2ea317883 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:42.851512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45faaa-69de-48eb-98d0-d6c2ea317883 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:42.867645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-ea45faaa-69de-48eb-98d0-d6c2ea317883 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:42.867645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:42.867645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:42.867645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:42.867645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:42.867645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:42.867645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:42.867645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:42.867645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:42.867645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:42.867645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:42.867645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:42.867645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:42.867645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:42.867645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:42.868566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:42.868566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:42.868566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:42.868566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:42.868566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:42.868566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:42.868566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:42.868566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:42.868566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:42.868566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:42.868566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:42.868566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:42.868566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:42.868566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:42.868566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:42.868566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:42.868566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:42.869445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:42.869445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:42.869445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:42.869445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:42.869445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:42.869445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:42.869445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:42.869445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:42.869445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 361/700] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:42.870431 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45faaa-69de-48eb-98d0-d6c2ea317883 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:42.903562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6ae6b-5a47-4476-89b6-db430f52acf6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:42.920886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:42.942773 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f584a-8c51-4899-af3e-7873fec92ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:42.944603 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f584a-8c51-4899-af3e-7873fec92ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:42.945234 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f584a-8c51-4899-af3e-7873fec92ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:42.945234 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f584a-8c51-4899-af3e-7873fec92ec9 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:42.945909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f584a-8c51-4899-af3e-7873fec92ec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:42.946318 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f584a-8c51-4899-af3e-7873fec92ec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:42.947612 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f584a-8c51-4899-af3e-7873fec92ec9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:42.964220 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 340/701] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:42.964220 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f584a-8c51-4899-af3e-7873fec92ec9 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:42.985647 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45faaa-69de-48eb-98d0-d6c2ea317883 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:43.022991 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2837a-bb1d-4c8b-b520-cc2ef5f97c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:43.023808 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2837a-bb1d-4c8b-b520-cc2ef5f97c30 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:43.024099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2837a-bb1d-4c8b-b520-cc2ef5f97c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:43.024388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2837a-bb1d-4c8b-b520-cc2ef5f97c30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:43.035659 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2837a-bb1d-4c8b-b520-cc2ef5f97c30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:43.037562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2837a-bb1d-4c8b-b520-cc2ef5f97c30 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:43.040000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:43.056592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-8da2837a-bb1d-4c8b-b520-cc2ef5f97c30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:43.056592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:43.056592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:43.056592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:43.056592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:43.056592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:43.056592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:43.056592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:43.056592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:43.056592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:43.056592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:43.056592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:43.056592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:43.056592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:43.056592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:43.057690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:43.057690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:43.057690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:43.057690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:43.057690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:43.057690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:43.057690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:43.057690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:43.057690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:43.057690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:43.057690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:43.057690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:43.057690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:43.057690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:43.057690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:43.057690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:43.057690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:43.058597 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:43.058597 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:43.058597 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:43.058597 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:43.058597 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 362/702] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:43.059137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3199-8543-4963-8326-766333b40dc1 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:43.059463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3199-8543-4963-8326-766333b40dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:43.059763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e3199-8543-4963-8326-766333b40dc1 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:43.062960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 363/703] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:43.062960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e3199-8543-4963-8326-766333b40dc1 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:43.085815 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f584a-8c51-4899-af3e-7873fec92ec9 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:43.129504 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fb78c-d907-4472-8031-1ae44eb29d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:43.130180 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fb78c-d907-4472-8031-1ae44eb29d67 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:43.130469 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fb78c-d907-4472-8031-1ae44eb29d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:43.130749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fb78c-d907-4472-8031-1ae44eb29d67 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:43.131964 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fb78c-d907-4472-8031-1ae44eb29d67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:43.132910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fb78c-d907-4472-8031-1ae44eb29d67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:43.134619 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fb78c-d907-4472-8031-1ae44eb29d67 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:43.156450 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 341/704] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:43.157262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fb78c-d907-4472-8031-1ae44eb29d67 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:43.163917 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:43.172231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e3199-8543-4963-8326-766333b40dc1 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:43.212017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a7052-eff4-4909-9ed7-87874fe5e556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:43.212697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a7052-eff4-4909-9ed7-87874fe5e556 None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:43.212985 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a7052-eff4-4909-9ed7-87874fe5e556 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:43.213261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a7052-eff4-4909-9ed7-87874fe5e556 None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:43.213967 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a7052-eff4-4909-9ed7-87874fe5e556 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:43.215440 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a7052-eff4-4909-9ed7-87874fe5e556 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:43.238012 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 364/705] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 22:23:43 2022] POST /identity/v3/users => generated 254 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:23:43.239654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91930869-0094-40d6-af61-c7c96fc7fe4d None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:43.239949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91930869-0094-40d6-af61-c7c96fc7fe4d None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:43.240455 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91930869-0094-40d6-af61-c7c96fc7fe4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:43.265341 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fb78c-d907-4472-8031-1ae44eb29d67 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:43.266131 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-91930869-0094-40d6-af61-c7c96fc7fe4d None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:43.267691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-91930869-0094-40d6-af61-c7c96fc7fe4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:43.304302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:43.309724 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34d8b6-4177-479f-ab5f-fd34a93b299b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:43.311138 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34d8b6-4177-479f-ab5f-fd34a93b299b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:43.312534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34d8b6-4177-479f-ab5f-fd34a93b299b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:43.312826 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34d8b6-4177-479f-ab5f-fd34a93b299b None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:43.313947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34d8b6-4177-479f-ab5f-fd34a93b299b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:43.314262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34d8b6-4177-479f-ab5f-fd34a93b299b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:43.319562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34d8b6-4177-479f-ab5f-fd34a93b299b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:43.336793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 342/706] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:43.337927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34d8b6-4177-479f-ab5f-fd34a93b299b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:43.375090 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 365/707] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:43.376340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91930869-0094-40d6-af61-c7c96fc7fe4d None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:43.486516 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34d8b6-4177-479f-ab5f-fd34a93b299b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:43.511161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:43.533741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54928dda-6d22-4ce0-9cac-00c7e09f63e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:43.534512 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54928dda-6d22-4ce0-9cac-00c7e09f63e1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:43.534695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54928dda-6d22-4ce0-9cac-00c7e09f63e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:43.535001 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54928dda-6d22-4ce0-9cac-00c7e09f63e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:43.535875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54928dda-6d22-4ce0-9cac-00c7e09f63e1 None admin] RBAC: Adding query filter params (role.id=2a71a120d6e34544aa5a2031ddf65ec8, scope.project.id=56da0975019c44409b53be473131143a, user.id=8ce35c43e8294f4cbc880f2219a6eb76) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:43.536191 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54928dda-6d22-4ce0-9cac-00c7e09f63e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:43.538217 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54928dda-6d22-4ce0-9cac-00c7e09f63e1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:43.539174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91930869-0094-40d6-af61-c7c96fc7fe4d None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:43.549719 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 343/708] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:23:43 2022] GET /identity/v3/role_assignments?user.id=8ce35c43e8294f4cbc880f2219a6eb76&scope.project.id=56da0975019c44409b53be473131143a&role.id=2a71a120d6e34544aa5a2031ddf65ec8 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:43.551373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54928dda-6d22-4ce0-9cac-00c7e09f63e1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:43.579470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f0e83-8bfa-47e5-be7e-bf266729a9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:43.580456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f0e83-8bfa-47e5-be7e-bf266729a9ae None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:43.580817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f0e83-8bfa-47e5-be7e-bf266729a9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:43.581217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f0e83-8bfa-47e5-be7e-bf266729a9ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:43.590133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f0e83-8bfa-47e5-be7e-bf266729a9ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:43.592093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f0e83-8bfa-47e5-be7e-bf266729a9ae None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:43.608942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-041f0e83-8bfa-47e5-be7e-bf266729a9ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:43.608942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:43.608942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:43.608942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:43.608942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:43.608942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:43.608942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:43.608942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:43.608942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:43.608942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:43.608942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:43.608942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:43.608942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:43.608942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:43.608942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:43.609920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:43.609920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:43.609920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:43.609920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:43.609920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:43.609920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:43.609920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:43.609920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:43.609920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:43.609920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:43.609920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:43.609920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:43.609920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:43.609920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:43.609920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:43.609920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:43.609920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:43.610882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:43.610882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:43.610882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:43.610882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:43.610882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:43.610882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:43.610882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:43.610882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:43.610882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 366/709] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:43.616677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f0e83-8bfa-47e5-be7e-bf266729a9ae None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:43.643187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:43.698059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54928dda-6d22-4ce0-9cac-00c7e09f63e1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:43.737223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f0e83-8bfa-47e5-be7e-bf266729a9ae None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:43.739182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4e8a1-453f-410d-9e49-a4842244fa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:43.740911 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4e8a1-453f-410d-9e49-a4842244fa2f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:43.741530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4e8a1-453f-410d-9e49-a4842244fa2f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:43.742069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4e8a1-453f-410d-9e49-a4842244fa2f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:43.742891 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4e8a1-453f-410d-9e49-a4842244fa2f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:43.744448 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4e8a1-453f-410d-9e49-a4842244fa2f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:43.760288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-63e4e8a1-453f-410d-9e49-a4842244fa2f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 22:23:43.760288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:43.760288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:23:43.760288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:23:43.760288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:23:43.760288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:23:43.760288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:23:43.760288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:23:43.760288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 22:23:43.760288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:43.760288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:23:43.760288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:43.760288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:43.760288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:43.760288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:43.760288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:43.761544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:43.761544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:43.761544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:43.761544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:43.761544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:43.761544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:43.761544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:43.761544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:23:43.761544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:23:43.761544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:23:43.761544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:23:43.761544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:43.761544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:43.761544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:43.761544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:43.761544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:43.762457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:43.762457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:43.762457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:43.762457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:43.762457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:43.762457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:43.762457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:43.762457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:43.762457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:43.762457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:43.762457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:43.762457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:23:43.762457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:23:43.762457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 22:23:43.762457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:43.763645 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 344/710] 10.222.0.43 () {56 vars in 1129 bytes} [Thu Jun 23 22:23:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:43.764828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abf83a-1fdc-42d4-955e-e6ad4d44ed05 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:43.765166 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abf83a-1fdc-42d4-955e-e6ad4d44ed05 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:43.765407 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abf83a-1fdc-42d4-955e-e6ad4d44ed05 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:43.766809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 345/711] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:43.772178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abf83a-1fdc-42d4-955e-e6ad4d44ed05 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:43.781488 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50523bc3-85b8-4207-b13b-a4fb4db38418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:43.782163 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50523bc3-85b8-4207-b13b-a4fb4db38418 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:43.782454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50523bc3-85b8-4207-b13b-a4fb4db38418 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:43.782738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50523bc3-85b8-4207-b13b-a4fb4db38418 None admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:43.783559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50523bc3-85b8-4207-b13b-a4fb4db38418 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:43.783879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50523bc3-85b8-4207-b13b-a4fb4db38418 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:43.785315 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50523bc3-85b8-4207-b13b-a4fb4db38418 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:43.799695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 367/712] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:43.800985 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cfd73-d0a4-4924-84c0-5fa907319cfc None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:43.801287 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cfd73-d0a4-4924-84c0-5fa907319cfc None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:43.801572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cfd73-d0a4-4924-84c0-5fa907319cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:43.812744 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:43.825226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-597cfd73-d0a4-4924-84c0-5fa907319cfc None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:43.826414 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-597cfd73-d0a4-4924-84c0-5fa907319cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:43.907837 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abf83a-1fdc-42d4-955e-e6ad4d44ed05 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:43.909819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 368/713] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:43.915744 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cfd73-d0a4-4924-84c0-5fa907319cfc None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:43.945723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:43.962726 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d2a49-fd8e-4bc3-8777-c8874a4d5d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:43.963552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d2a49-fd8e-4bc3-8777-c8874a4d5d4f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:43.963967 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d2a49-fd8e-4bc3-8777-c8874a4d5d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:43.964260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d2a49-fd8e-4bc3-8777-c8874a4d5d4f None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:43.965032 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d2a49-fd8e-4bc3-8777-c8874a4d5d4f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:43.965352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d2a49-fd8e-4bc3-8777-c8874a4d5d4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:43.966734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d2a49-fd8e-4bc3-8777-c8874a4d5d4f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:43.972127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 346/714] 10.222.0.43 () {56 vars in 1124 bytes} [Thu Jun 23 22:23:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:43.972936 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d2a49-fd8e-4bc3-8777-c8874a4d5d4f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:44.038465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cfd73-d0a4-4924-84c0-5fa907319cfc None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:44.074270 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:44.082772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d68ae-ca94-4332-857a-db57e8b1e88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:44.084198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d68ae-ca94-4332-857a-db57e8b1e88c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:44.084511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d68ae-ca94-4332-857a-db57e8b1e88c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:44.084777 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d68ae-ca94-4332-857a-db57e8b1e88c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:44.085628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d68ae-ca94-4332-857a-db57e8b1e88c None admin] RBAC: Adding query filter params (role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.project.id=56da0975019c44409b53be473131143a, user.id=35d686923ef541819de3c77847d38511) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:44.085949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d68ae-ca94-4332-857a-db57e8b1e88c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:44.086425 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d2a49-fd8e-4bc3-8777-c8874a4d5d4f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:44.087461 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d68ae-ca94-4332-857a-db57e8b1e88c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:44.095036 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 369/715] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:23:43 2022] GET /identity/v3/role_assignments?user.id=35d686923ef541819de3c77847d38511&scope.project.id=56da0975019c44409b53be473131143a&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:44.095897 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d68ae-ca94-4332-857a-db57e8b1e88c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:44.166617 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b67f7d-6add-499f-adb1-d40eca022b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:44.167410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b67f7d-6add-499f-adb1-d40eca022b95 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:44.167704 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b67f7d-6add-499f-adb1-d40eca022b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:44.167988 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b67f7d-6add-499f-adb1-d40eca022b95 None admin] PATH_INFO: `/v3/services/network` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:44.169201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b67f7d-6add-499f-adb1-d40eca022b95 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:44.170642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b67f7d-6add-499f-adb1-d40eca022b95 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:44.184892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a7b67f7d-6add-499f-adb1-d40eca022b95 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 22:23:44.184892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:44.184892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:23:44.184892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:23:44.184892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:23:44.184892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:23:44.184892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:23:44.184892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:23:44.184892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 22:23:44.184892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:44.184892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:23:44.184892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:44.184892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:44.184892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:44.184892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:44.184892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:44.184892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:44.186006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:44.186006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:44.186006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:44.186006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:44.186006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:44.186006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:44.186006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:23:44.186006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:23:44.186006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:23:44.186006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:23:44.186006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:44.186006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:44.186006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:44.186006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:44.186006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:44.186006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:44.186873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:44.186873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:44.186873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:44.186873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:44.186873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:44.186873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:44.186873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:44.186873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:44.186873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:44.186873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:44.186873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:23:44.186873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:23:44.186873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 22:23:44.186873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:44.191453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 347/716] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:43 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:44.196175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b67f7d-6add-499f-adb1-d40eca022b95 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:44.291347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:44.300880 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d68ae-ca94-4332-857a-db57e8b1e88c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:44.363605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e3764-b3ee-495a-88b7-549671830e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:44.364282 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e3764-b3ee-495a-88b7-549671830e30 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:44.364568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e3764-b3ee-495a-88b7-549671830e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:44.364849 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e3764-b3ee-495a-88b7-549671830e30 None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:44.365810 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e3764-b3ee-495a-88b7-549671830e30 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:44.366139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e3764-b3ee-495a-88b7-549671830e30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:44.368600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e3764-b3ee-495a-88b7-549671830e30 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:44.378842 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 370/717] 10.222.0.43 () {56 vars in 1124 bytes} [Thu Jun 23 22:23:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:44.395744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b67f7d-6add-499f-adb1-d40eca022b95 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:44.451585 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185496c6-1185-40e5-b291-1cf30678136e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:44.452189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185496c6-1185-40e5-b291-1cf30678136e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:44.452482 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185496c6-1185-40e5-b291-1cf30678136e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:44.452768 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185496c6-1185-40e5-b291-1cf30678136e None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:44.453496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185496c6-1185-40e5-b291-1cf30678136e None admin] RBAC: Adding query filter params (name=network) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:44.453812 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185496c6-1185-40e5-b291-1cf30678136e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:44.455232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185496c6-1185-40e5-b291-1cf30678136e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:44.463743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 348/718] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 22:23:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:44.470179 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e3764-b3ee-495a-88b7-549671830e30 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:44.582080 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:44.631680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e3764-b3ee-495a-88b7-549671830e30 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:44.669997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc5192-663b-4932-b9e8-9f29c61fa36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:44.670683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc5192-663b-4932-b9e8-9f29c61fa36f None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:44.674879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc5192-663b-4932-b9e8-9f29c61fa36f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:44.674879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc5192-663b-4932-b9e8-9f29c61fa36f None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:44.674879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc5192-663b-4932-b9e8-9f29c61fa36f None admin] RBAC: Adding query filter params (type=network) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:44.674879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc5192-663b-4932-b9e8-9f29c61fa36f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:44.674879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc5192-663b-4932-b9e8-9f29c61fa36f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:44.680610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 371/719] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 22:23:44 2022] GET /identity/v3/services?type=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:45.660157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571a537-1f76-44c0-870e-e4f77e77f426 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:45.660683 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571a537-1f76-44c0-870e-e4f77e77f426 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:45.660764 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571a537-1f76-44c0-870e-e4f77e77f426 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:45.662866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 349/720] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:45.668351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7e394-9469-4cd1-bead-b6a35db49715 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:45.668792 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7e394-9469-4cd1-bead-b6a35db49715 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:45.668972 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7e394-9469-4cd1-bead-b6a35db49715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:45.691996 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-d8b7e394-9469-4cd1-bead-b6a35db49715 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:45.693207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d8b7e394-9469-4cd1-bead-b6a35db49715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:45.790163 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 372/721] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 22:23:45.825426 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571a537-1f76-44c0-870e-e4f77e77f426 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:45.967644 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571a537-1f76-44c0-870e-e4f77e77f426 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:45.986207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:46.012542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66801aa2-0cc7-4507-bea3-eaf19c655352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:46.013338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66801aa2-0cc7-4507-bea3-eaf19c655352 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:46.013629 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66801aa2-0cc7-4507-bea3-eaf19c655352 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:46.013906 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66801aa2-0cc7-4507-bea3-eaf19c655352 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:46.035863 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66801aa2-0cc7-4507-bea3-eaf19c655352 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:46.043533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-66801aa2-0cc7-4507-bea3-eaf19c655352 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:46.043533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:46.043533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:46.043533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:46.043533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:46.043533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:46.043533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:46.043533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:46.043533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:46.043533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:46.043533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:46.043533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:46.043533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:46.043533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:46.043533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:46.043533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:46.044457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:46.044457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:46.044457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:46.044457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:46.044457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:46.044457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:46.044457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:46.044457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:46.044457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:46.044457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:46.044457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:46.044457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:46.044457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:46.044457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:46.044457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:46.044457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:46.045304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:46.045304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:46.045304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:46.045304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:46.045304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:46.045304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:46.045304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:46.045304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:46.045304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:46.045304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:46.045304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:46.045304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:46.049371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 350/722] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:46.052186 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7e394-9469-4cd1-bead-b6a35db49715 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:46.133388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64730a6b-cb50-48f1-a660-7e32f36a767f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:46.133682 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64730a6b-cb50-48f1-a660-7e32f36a767f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:46.134095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64730a6b-cb50-48f1-a660-7e32f36a767f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:46.135098 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:46.136641 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 351/723] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:23:46.145173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397386b-d70c-452c-8357-8b7c7f8224b5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:46.145642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397386b-d70c-452c-8357-8b7c7f8224b5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:46.146100 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397386b-d70c-452c-8357-8b7c7f8224b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:46.177003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5397386b-d70c-452c-8357-8b7c7f8224b5 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:46.180266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5397386b-d70c-452c-8357-8b7c7f8224b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:46.213592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7e394-9469-4cd1-bead-b6a35db49715 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:46.261713 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 352/724] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:46.279687 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86824d1e-dd9a-439a-9972-39a8a2c99da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:46.280391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86824d1e-dd9a-439a-9972-39a8a2c99da2 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:46.280686 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86824d1e-dd9a-439a-9972-39a8a2c99da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:46.280971 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86824d1e-dd9a-439a-9972-39a8a2c99da2 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:46.281795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86824d1e-dd9a-439a-9972-39a8a2c99da2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:46.282115 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86824d1e-dd9a-439a-9972-39a8a2c99da2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:46.283539 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86824d1e-dd9a-439a-9972-39a8a2c99da2 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:46.296619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 373/725] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:23:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:46.300598 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397386b-d70c-452c-8357-8b7c7f8224b5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:46.307987 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86824d1e-dd9a-439a-9972-39a8a2c99da2 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:46.404607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397386b-d70c-452c-8357-8b7c7f8224b5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:46.406632 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:46.442435 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbcae28-2f1f-4f7b-8339-7de674099d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:46.444295 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcae28-2f1f-4f7b-8339-7de674099d55 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:46.444804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcae28-2f1f-4f7b-8339-7de674099d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:46.445379 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcae28-2f1f-4f7b-8339-7de674099d55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:46.456674 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbcae28-2f1f-4f7b-8339-7de674099d55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:46.464846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4cbcae28-2f1f-4f7b-8339-7de674099d55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:23:46.464846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:46.464846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:46.464846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:46.464846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:46.464846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:46.464846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:46.464846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:46.464846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:46.464846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:46.464846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:46.464846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:46.464846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:46.464846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:46.464846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:46.464846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:46.465804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:46.465804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:46.465804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:46.465804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:46.465804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:46.465804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:46.465804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:46.465804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:46.465804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:46.465804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:46.465804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:46.465804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:46.465804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:46.465804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:46.465804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:46.465804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:46.466688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:46.466688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:46.466688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:46.466688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:46.466688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:46.466688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:46.466688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:46.466688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:46.466688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:46.466688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:46.466688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:23:46.466688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:46.469464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 353/726] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:23:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:46.475923 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbcae28-2f1f-4f7b-8339-7de674099d55 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:46.538711 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86824d1e-dd9a-439a-9972-39a8a2c99da2 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:46.570485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:46.587950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbcae28-2f1f-4f7b-8339-7de674099d55 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:46.607324 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b48330-bfb6-41b0-bc48-5e3b1e48ffcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:46.608194 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b48330-bfb6-41b0-bc48-5e3b1e48ffcf None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:46.608486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b48330-bfb6-41b0-bc48-5e3b1e48ffcf None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:46.608772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b48330-bfb6-41b0-bc48-5e3b1e48ffcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:46.626792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018dfe00-39a3-472b-b61f-23377ff16594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:46.627841 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018dfe00-39a3-472b-b61f-23377ff16594 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:46.628356 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018dfe00-39a3-472b-b61f-23377ff16594 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:46.628857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018dfe00-39a3-472b-b61f-23377ff16594 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:46.629810 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018dfe00-39a3-472b-b61f-23377ff16594 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:46.630336 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018dfe00-39a3-472b-b61f-23377ff16594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:46.632353 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018dfe00-39a3-472b-b61f-23377ff16594 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:46.639271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b48330-bfb6-41b0-bc48-5e3b1e48ffcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:46.640767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b48330-bfb6-41b0-bc48-5e3b1e48ffcf None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:46.642807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 354/727] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:46.649315 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018dfe00-39a3-472b-b61f-23377ff16594 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:46.680248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-75b48330-bfb6-41b0-bc48-5e3b1e48ffcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:46.680248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:46.680248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:46.680248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:46.680248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:46.680248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:46.680248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:46.680248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:46.680248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:46.680248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:46.680248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:46.680248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:46.680248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:46.680248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:46.680248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:46.681576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:46.681576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:46.681576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:46.681576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:46.681576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:46.681576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:46.681576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:46.681576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:46.681576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:46.681576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:46.681576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:46.681576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:46.681576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:46.681576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:46.681576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:46.681576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:46.681576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:46.682469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:46.682469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:46.682469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:46.682469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:46.682469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 374/728] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:46.683492 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d65725-5674-4424-ab34-e4b93edd9bb3 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:46.683834 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d65725-5674-4424-ab34-e4b93edd9bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:46.684051 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d65725-5674-4424-ab34-e4b93edd9bb3 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:46.689272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 375/729] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:46.689993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d65725-5674-4424-ab34-e4b93edd9bb3 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:46.691308 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:46.762766 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018dfe00-39a3-472b-b61f-23377ff16594 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:46.799498 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a830e6-0eba-4872-b187-4b2885ec720d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:46.800241 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a830e6-0eba-4872-b187-4b2885ec720d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:46.800550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a830e6-0eba-4872-b187-4b2885ec720d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:46.800823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a830e6-0eba-4872-b187-4b2885ec720d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:46.810660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d65725-5674-4424-ab34-e4b93edd9bb3 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:46.811435 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a830e6-0eba-4872-b187-4b2885ec720d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:46.812952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a830e6-0eba-4872-b187-4b2885ec720d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:46.831411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a2a830e6-0eba-4872-b187-4b2885ec720d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:46.831411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:46.831411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:46.831411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:46.831411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:46.831411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:46.831411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:46.831411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:46.831411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:46.831411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:46.831411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:46.831411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:46.831411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:46.831411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:46.831411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:46.832364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:46.832364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:46.832364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:46.832364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:46.832364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:46.832364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:46.832364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:46.832364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:46.832364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:46.832364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:46.832364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:46.832364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:46.832364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:46.832364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:46.832364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:46.832364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:46.832364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:46.833260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:46.833260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:46.833260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:46.833260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:46.838138 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 355/730] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:23:46.839509 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e3b45-54ea-4b49-a1d4-db596436ad24 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:46.839811 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e3b45-54ea-4b49-a1d4-db596436ad24 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:46.840107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e3b45-54ea-4b49-a1d4-db596436ad24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:46.852770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98c2bd-539c-460c-a714-7636d561b646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:46.853775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98c2bd-539c-460c-a714-7636d561b646 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:46.854285 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98c2bd-539c-460c-a714-7636d561b646 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:46.854762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98c2bd-539c-460c-a714-7636d561b646 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:46.855699 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98c2bd-539c-460c-a714-7636d561b646 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:46.856220 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98c2bd-539c-460c-a714-7636d561b646 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:46.857714 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-187e3b45-54ea-4b49-a1d4-db596436ad24 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:46.858152 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98c2bd-539c-460c-a714-7636d561b646 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:46.858891 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-187e3b45-54ea-4b49-a1d4-db596436ad24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:46.872483 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 376/731] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:46.873724 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98c2bd-539c-460c-a714-7636d561b646 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:46.905013 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:46.937017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 356/732] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:46.938163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e3b45-54ea-4b49-a1d4-db596436ad24 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:46.947324 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:46.985861 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98c2bd-539c-460c-a714-7636d561b646 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:47.025817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4995491-e42a-41db-abe1-036f71fd3833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:47.026500 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4995491-e42a-41db-abe1-036f71fd3833 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:47.026785 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4995491-e42a-41db-abe1-036f71fd3833 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:47.027100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4995491-e42a-41db-abe1-036f71fd3833 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:47.027826 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4995491-e42a-41db-abe1-036f71fd3833 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:47.028136 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4995491-e42a-41db-abe1-036f71fd3833 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:47.029515 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4995491-e42a-41db-abe1-036f71fd3833 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:47.045314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 377/733] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:47.046734 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e347411-b2fe-43f5-80a1-bd00ed43b313 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:47.047022 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e347411-b2fe-43f5-80a1-bd00ed43b313 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:47.047308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e347411-b2fe-43f5-80a1-bd00ed43b313 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:47.048553 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 378/734] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:47.049240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e347411-b2fe-43f5-80a1-bd00ed43b313 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:47.050602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e3b45-54ea-4b49-a1d4-db596436ad24 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:47.088879 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058afc4a-acd3-4c3f-80d0-c4a01e741b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:47.089691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058afc4a-acd3-4c3f-80d0-c4a01e741b97 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:47.089979 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058afc4a-acd3-4c3f-80d0-c4a01e741b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:47.090259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058afc4a-acd3-4c3f-80d0-c4a01e741b97 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:47.097440 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058afc4a-acd3-4c3f-80d0-c4a01e741b97 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:47.098875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058afc4a-acd3-4c3f-80d0-c4a01e741b97 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:47.115008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-058afc4a-acd3-4c3f-80d0-c4a01e741b97 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:23:47.115008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:47.115008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:47.115008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:47.115008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:47.115008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:47.115008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:47.115008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:47.115008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:47.115008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:47.115008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:47.115008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:47.115008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:47.115008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:47.115008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:47.115008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:47.116882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:47.116882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:47.116882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:47.116882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:47.116882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:47.116882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:47.116882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:47.116882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:47.116882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:47.116882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:47.116882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:47.116882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:47.116882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:47.116882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:47.116882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:47.116882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:47.116882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:47.116882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:47.117835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:47.117835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:47.117835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:47.117835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:47.117835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:47.117835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:47.117835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:47.117835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:47.117835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:23:47.117835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:47.117835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 357/735] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 22:23:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:47.118555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058afc4a-acd3-4c3f-80d0-c4a01e741b97 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:47.161746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:47.176741 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e347411-b2fe-43f5-80a1-bd00ed43b313 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:47.193552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:47.215221 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e4b78-0057-45b2-9a1b-ea40dfca54e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:47.216024 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e4b78-0057-45b2-9a1b-ea40dfca54e0 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:47.216310 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e4b78-0057-45b2-9a1b-ea40dfca54e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:47.216592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e4b78-0057-45b2-9a1b-ea40dfca54e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:47.227754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e4b78-0057-45b2-9a1b-ea40dfca54e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:47.230120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058afc4a-acd3-4c3f-80d0-c4a01e741b97 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:47.235326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-1c8e4b78-0057-45b2-9a1b-ea40dfca54e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:23:47.235326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:47.235326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:47.235326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:47.235326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:47.235326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:47.235326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:47.235326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:47.235326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:47.235326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:47.235326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:47.235326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:47.235326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:47.235326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:47.235326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:47.235326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:47.236268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:47.236268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:47.236268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:47.236268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:47.236268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:47.236268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:47.236268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:47.236268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:47.236268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:47.236268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:47.236268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:47.236268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:47.236268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:47.236268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:47.236268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:47.236268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:47.237153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:47.237153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:47.237153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:47.237153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:47.237153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:47.237153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:47.237153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:47.237153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:47.237153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:47.237153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:47.237153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:23:47.237153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:47.237153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 379/736] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:23:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:47.238211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939e1b0-ad17-4f6e-a34a-49f2377c7d6a None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:47.238437 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939e1b0-ad17-4f6e-a34a-49f2377c7d6a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:47.238719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939e1b0-ad17-4f6e-a34a-49f2377c7d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:47.255984 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-a939e1b0-ad17-4f6e-a34a-49f2377c7d6a None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:47.257200 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-a939e1b0-ad17-4f6e-a34a-49f2377c7d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:47.271971 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73583ce7-20d7-4097-937d-a7f0570e7a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:47.273122 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73583ce7-20d7-4097-937d-a7f0570e7a9d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:47.273415 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73583ce7-20d7-4097-937d-a7f0570e7a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:47.273698 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73583ce7-20d7-4097-937d-a7f0570e7a9d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:47.282268 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73583ce7-20d7-4097-937d-a7f0570e7a9d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:47.283739 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73583ce7-20d7-4097-937d-a7f0570e7a9d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:47.298496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-73583ce7-20d7-4097-937d-a7f0570e7a9d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:23:47.298496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:47.298496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:47.298496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:47.298496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:47.298496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:47.298496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:47.298496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:47.298496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:47.298496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:47.298496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:47.298496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:47.298496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:47.298496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:47.298496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:47.298496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:47.299666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:47.299666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:47.299666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:47.299666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:47.299666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:47.299666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:47.299666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:47.299666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:47.299666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:47.299666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:47.299666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:47.299666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:47.299666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:47.299666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:47.299666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:47.299666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:47.299666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:47.299666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:47.300921 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:47.300921 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:47.300921 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:47.300921 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:47.300921 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:47.300921 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:47.300921 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:47.300921 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:47.300921 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:23:47.300921 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:47.304853 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 358/737] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:23:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:47.305584 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73583ce7-20d7-4097-937d-a7f0570e7a9d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:47.332627 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 380/738] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:47.333831 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939e1b0-ad17-4f6e-a34a-49f2377c7d6a None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:47.420273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:47.443154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939e1b0-ad17-4f6e-a34a-49f2377c7d6a None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:47.476622 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82c89c-dd5b-4d14-9743-8bcc16c4f8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:47.477298 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82c89c-dd5b-4d14-9743-8bcc16c4f8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:47.477729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82c89c-dd5b-4d14-9743-8bcc16c4f8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:47.477881 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce82c89c-dd5b-4d14-9743-8bcc16c4f8d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:47.478595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce82c89c-dd5b-4d14-9743-8bcc16c4f8d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:47.478902 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce82c89c-dd5b-4d14-9743-8bcc16c4f8d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:47.480296 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce82c89c-dd5b-4d14-9743-8bcc16c4f8d8 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:47.488475 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 381/739] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:47.489248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82c89c-dd5b-4d14-9743-8bcc16c4f8d8 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:47.527726 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73583ce7-20d7-4097-937d-a7f0570e7a9d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:47.552512 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:47.561945 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e609d0e-30ed-4f31-a91b-fa907b9012bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:47.562616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e609d0e-30ed-4f31-a91b-fa907b9012bf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:47.562904 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e609d0e-30ed-4f31-a91b-fa907b9012bf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:47.563225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e609d0e-30ed-4f31-a91b-fa907b9012bf None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:47.564064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e609d0e-30ed-4f31-a91b-fa907b9012bf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:47.564374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e609d0e-30ed-4f31-a91b-fa907b9012bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:47.565795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e609d0e-30ed-4f31-a91b-fa907b9012bf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:47.583835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 359/740] 10.222.0.43 () {56 vars in 1149 bytes} [Thu Jun 23 22:23:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:47.585551 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2f8f6-8db5-43e5-bd1a-3b4424bc17f1 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:47.585844 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2f8f6-8db5-43e5-bd1a-3b4424bc17f1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:47.586124 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2f8f6-8db5-43e5-bd1a-3b4424bc17f1 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:47.587671 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 360/741] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:47.588579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2f8f6-8db5-43e5-bd1a-3b4424bc17f1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:47.597307 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce82c89c-dd5b-4d14-9743-8bcc16c4f8d8 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:47.631099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444ec48-cf7c-438f-bd50-43b4a28d3924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:47.631769 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444ec48-cf7c-438f-bd50-43b4a28d3924 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:47.632053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444ec48-cf7c-438f-bd50-43b4a28d3924 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:47.632335 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444ec48-cf7c-438f-bd50-43b4a28d3924 None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:47.633168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444ec48-cf7c-438f-bd50-43b4a28d3924 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:47.633476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444ec48-cf7c-438f-bd50-43b4a28d3924 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:47.634894 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444ec48-cf7c-438f-bd50-43b4a28d3924 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:47.644520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:47.653856 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 382/742] 10.222.0.43 () {56 vars in 1135 bytes} [Thu Jun 23 22:23:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:47.654851 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444ec48-cf7c-438f-bd50-43b4a28d3924 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:47.689723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2f8f6-8db5-43e5-bd1a-3b4424bc17f1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:47.722883 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e073b4c-f100-40d7-b258-706b86f01046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:47.723573 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e073b4c-f100-40d7-b258-706b86f01046 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:47.723861 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e073b4c-f100-40d7-b258-706b86f01046 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:47.724150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e073b4c-f100-40d7-b258-706b86f01046 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:47.725359 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e073b4c-f100-40d7-b258-706b86f01046 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:47.726722 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e073b4c-f100-40d7-b258-706b86f01046 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:47.741708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 361/743] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 22:23:47 2022] POST /identity/v3/services => generated 251 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:23:47.742713 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e073b4c-f100-40d7-b258-706b86f01046 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:47.758214 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444ec48-cf7c-438f-bd50-43b4a28d3924 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:47.795667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:47.813472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0651c4-2cee-47fb-860f-a2bb628c37bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:47.814206 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0651c4-2cee-47fb-860f-a2bb628c37bb None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:47.814499 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0651c4-2cee-47fb-860f-a2bb628c37bb None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:47.814779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0651c4-2cee-47fb-860f-a2bb628c37bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:47.828079 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0651c4-2cee-47fb-860f-a2bb628c37bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:47.829824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0651c4-2cee-47fb-860f-a2bb628c37bb None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:47.851684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-dc0651c4-2cee-47fb-860f-a2bb628c37bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:47.851684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:47.851684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:47.851684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:47.851684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:47.851684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:47.851684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:47.851684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:47.851684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:47.851684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:47.851684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:47.851684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:47.851684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:47.851684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:47.851684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:47.852700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:47.852700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:47.852700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:47.852700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:47.852700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:47.852700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:47.852700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:47.852700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:47.852700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:47.852700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:47.852700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:47.852700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:47.852700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:47.852700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:47.852700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:47.852700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:47.852700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:47.853576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:47.853576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:47.853576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:47.853576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:47.853576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 383/744] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:47.854444 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e538f02-3ee0-494f-ab6f-c3aa3a2c3e42 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:47.854742 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e538f02-3ee0-494f-ab6f-c3aa3a2c3e42 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:47.855051 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e538f02-3ee0-494f-ab6f-c3aa3a2c3e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:47.863659 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e073b4c-f100-40d7-b258-706b86f01046 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:47.873604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-8e538f02-3ee0-494f-ab6f-c3aa3a2c3e42 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:47.875057 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8e538f02-3ee0-494f-ab6f-c3aa3a2c3e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:47.903563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca973fc-1af3-4568-b455-b86d4f336b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:47.904536 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca973fc-1af3-4568-b455-b86d4f336b15 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:47.904822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca973fc-1af3-4568-b455-b86d4f336b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:47.905100 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca973fc-1af3-4568-b455-b86d4f336b15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:47.926097 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:47.953494 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 384/745] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 22:23:47.954338 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e538f02-3ee0-494f-ab6f-c3aa3a2c3e42 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:47.996882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca973fc-1af3-4568-b455-b86d4f336b15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:47.998677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca973fc-1af3-4568-b455-b86d4f336b15 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:48.045545 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:48.076087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e538f02-3ee0-494f-ab6f-c3aa3a2c3e42 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:48.107949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 362/746] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:47 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:48.107949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca973fc-1af3-4568-b455-b86d4f336b15 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:48.119224 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c36822-9212-4dca-bbe1-2adc90f14637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:48.120147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c36822-9212-4dca-bbe1-2adc90f14637 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:48.120612 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c36822-9212-4dca-bbe1-2adc90f14637 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:48.121086 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c36822-9212-4dca-bbe1-2adc90f14637 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:48.181026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:48.208080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca973fc-1af3-4568-b455-b86d4f336b15 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:48.211387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c36822-9212-4dca-bbe1-2adc90f14637 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:48.212870 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c36822-9212-4dca-bbe1-2adc90f14637 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:48.245182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffd5ee-71e5-4eed-a9af-92956240fea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:48.245841 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffd5ee-71e5-4eed-a9af-92956240fea7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:48.246126 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffd5ee-71e5-4eed-a9af-92956240fea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:48.246409 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffd5ee-71e5-4eed-a9af-92956240fea7 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:48.247152 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffd5ee-71e5-4eed-a9af-92956240fea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:48.247463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffd5ee-71e5-4eed-a9af-92956240fea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:48.248820 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffd5ee-71e5-4eed-a9af-92956240fea7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:48.260677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 363/747] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:48.265271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffd5ee-71e5-4eed-a9af-92956240fea7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:48.267496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:48.322324 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 385/748] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:47 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 22:23:48.323180 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c36822-9212-4dca-bbe1-2adc90f14637 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:48.372278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffd5ee-71e5-4eed-a9af-92956240fea7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:48.407317 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:48.407510 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d03b85-726e-47d4-9313-797691b77ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:48.408186 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d03b85-726e-47d4-9313-797691b77ad4 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:48.408479 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d03b85-726e-47d4-9313-797691b77ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:48.408760 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d03b85-726e-47d4-9313-797691b77ad4 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:48.409491 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d03b85-726e-47d4-9313-797691b77ad4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:48.412534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d03b85-726e-47d4-9313-797691b77ad4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:48.425669 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 364/749] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 22:23:48 2022] POST /identity/v3/services => generated 240 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:23:48.426712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d03b85-726e-47d4-9313-797691b77ad4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:48.449205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c36822-9212-4dca-bbe1-2adc90f14637 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:48.507131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:48.518690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d7af1-e42e-4377-8ede-5ced007f7be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:48.520289 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d7af1-e42e-4377-8ede-5ced007f7be7 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:48.521824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d7af1-e42e-4377-8ede-5ced007f7be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:48.522354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d7af1-e42e-4377-8ede-5ced007f7be7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:48.538758 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d7af1-e42e-4377-8ede-5ced007f7be7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:48.540692 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d7af1-e42e-4377-8ede-5ced007f7be7 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:48.555736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d03b85-726e-47d4-9313-797691b77ad4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:48.573009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-1f8d7af1-e42e-4377-8ede-5ced007f7be7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:48.573009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:48.573009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:48.573009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:48.573009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:48.573009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:48.573009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:48.573009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:48.573009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:48.573009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:48.573009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:48.573009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:48.573009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:48.573009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:48.573009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:48.573978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:48.573978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:48.573978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:48.573978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:48.573978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:48.573978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:48.573978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:48.573978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:48.573978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:48.573978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:48.573978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:48.573978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:48.573978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:48.573978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:48.573978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:48.573978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:48.573978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:48.574847 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:48.574847 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:48.574847 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:48.574847 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:48.581117 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 386/750] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:48.582064 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d7af1-e42e-4377-8ede-5ced007f7be7 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:48.603700 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d225a18-54d8-4bef-b618-b5c66f0515bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:48.604503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d225a18-54d8-4bef-b618-b5c66f0515bd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:48.604974 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d225a18-54d8-4bef-b618-b5c66f0515bd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:48.605269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d225a18-54d8-4bef-b618-b5c66f0515bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:48.615009 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d225a18-54d8-4bef-b618-b5c66f0515bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:48.617239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d225a18-54d8-4bef-b618-b5c66f0515bd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:48.634326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-8d225a18-54d8-4bef-b618-b5c66f0515bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:23:48.634326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:48.634326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:48.634326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:48.634326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:48.634326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:48.634326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:48.634326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:48.634326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:48.634326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:48.634326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:48.634326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:48.634326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:48.634326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:48.634326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:48.634326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:48.635517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:48.635517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:48.635517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:48.635517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:48.635517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:48.635517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:48.635517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:48.635517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:48.635517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:48.635517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:48.635517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:48.635517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:48.635517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:48.635517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:48.635517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:48.635517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:48.635517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:48.635517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:48.636654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:48.636654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:48.636654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:48.636654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:48.636654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:48.636654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:48.636654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:48.636654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:48.636654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:23:48.636654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:48.638594 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 365/751] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Jun 23 22:23:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:48.639637 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d225a18-54d8-4bef-b618-b5c66f0515bd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:48.727283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d7af1-e42e-4377-8ede-5ced007f7be7 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:48.752892 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:48.763474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d225a18-54d8-4bef-b618-b5c66f0515bd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:48.765590 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6bb14-416c-4bde-b977-ca8e9bfe772d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:48.766488 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6bb14-416c-4bde-b977-ca8e9bfe772d None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:48.766872 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6bb14-416c-4bde-b977-ca8e9bfe772d None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:48.767277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6bb14-416c-4bde-b977-ca8e9bfe772d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:48.778273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6bb14-416c-4bde-b977-ca8e9bfe772d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:48.779909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6bb14-416c-4bde-b977-ca8e9bfe772d None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:48.794305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:48.799054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-85d6bb14-416c-4bde-b977-ca8e9bfe772d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:48.799054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:48.799054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:48.799054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:48.799054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:48.799054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:48.799054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:48.799054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:48.799054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:48.799054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:48.799054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:48.799054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:48.799054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:48.799054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:48.799054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:48.800137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:48.800137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:48.800137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:48.800137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:48.800137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:48.800137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:48.800137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:48.800137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:48.800137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:48.800137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:48.800137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:48.800137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:48.800137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:48.800137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:48.800137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:48.800137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:48.800137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:48.801071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:48.801071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:48.801071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:48.801071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:48.801831 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 387/752] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:48.803347 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6bb14-416c-4bde-b977-ca8e9bfe772d None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:48.811282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3ae54-e4ff-45c5-8668-a24b8015e22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:48.812162 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3ae54-e4ff-45c5-8668-a24b8015e22a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:48.813011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3ae54-e4ff-45c5-8668-a24b8015e22a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:48.813469 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3ae54-e4ff-45c5-8668-a24b8015e22a None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:48.814353 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3ae54-e4ff-45c5-8668-a24b8015e22a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:48.814986 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3ae54-e4ff-45c5-8668-a24b8015e22a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:48.817160 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3ae54-e4ff-45c5-8668-a24b8015e22a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:48.839573 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 366/753] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:48.840142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3ae54-e4ff-45c5-8668-a24b8015e22a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:48.917808 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6bb14-416c-4bde-b977-ca8e9bfe772d None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:48.957969 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33b6c8-704e-4cb6-8f03-11d2dff167b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:48.959598 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33b6c8-704e-4cb6-8f03-11d2dff167b1 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:48.960636 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33b6c8-704e-4cb6-8f03-11d2dff167b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:48.961148 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33b6c8-704e-4cb6-8f03-11d2dff167b1 None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:48.962471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33b6c8-704e-4cb6-8f03-11d2dff167b1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:48.963287 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33b6c8-704e-4cb6-8f03-11d2dff167b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:48.965603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33b6c8-704e-4cb6-8f03-11d2dff167b1 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:48.972809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3ae54-e4ff-45c5-8668-a24b8015e22a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:48.990001 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 388/754] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Jun 23 22:23:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:48.991428 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33b6c8-704e-4cb6-8f03-11d2dff167b1 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:49.014064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2ad70-bba2-41e0-b17d-03fa5b3e0a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:49.014791 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2ad70-bba2-41e0-b17d-03fa5b3e0a6d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:49.015230 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2ad70-bba2-41e0-b17d-03fa5b3e0a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:49.015561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2ad70-bba2-41e0-b17d-03fa5b3e0a6d None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:49.016281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2ad70-bba2-41e0-b17d-03fa5b3e0a6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:49.016588 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2ad70-bba2-41e0-b17d-03fa5b3e0a6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:49.018009 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2ad70-bba2-41e0-b17d-03fa5b3e0a6d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:49.031540 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 367/755] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:49.032579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2ad70-bba2-41e0-b17d-03fa5b3e0a6d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:49.035692 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:49.088956 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:49.120777 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33b6c8-704e-4cb6-8f03-11d2dff167b1 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:49.160861 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2ad70-bba2-41e0-b17d-03fa5b3e0a6d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:49.176966 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb62335-9f07-4631-8765-2ea07fcecf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:49.178860 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb62335-9f07-4631-8765-2ea07fcecf3e None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:49.179414 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb62335-9f07-4631-8765-2ea07fcecf3e None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:49.179964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb62335-9f07-4631-8765-2ea07fcecf3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:49.190341 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb62335-9f07-4631-8765-2ea07fcecf3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:49.192476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb62335-9f07-4631-8765-2ea07fcecf3e None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:49.198040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb8999-53df-4b90-a7a1-86159f5d2309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:49.198831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb8999-53df-4b90-a7a1-86159f5d2309 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:49.199159 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb8999-53df-4b90-a7a1-86159f5d2309 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:49.199436 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb8999-53df-4b90-a7a1-86159f5d2309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:49.211301 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb8999-53df-4b90-a7a1-86159f5d2309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:49.212377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-dfb62335-9f07-4631-8765-2ea07fcecf3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:49.212377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:49.212377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:49.212377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:49.212377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:49.212377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:49.212377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:49.212377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:49.212377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:49.212377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:49.212377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:49.212377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:49.212377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:49.212377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:49.212377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:49.213340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:49.213340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:49.213340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:49.213340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:49.213340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:49.213340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:49.213340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:49.213340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:49.213340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:49.213340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:49.213340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:49.213340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:49.213340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:49.213340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:49.213340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:49.213340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:49.213340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:49.214244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:49.214244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:49.214244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:49.214244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:49.214244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:49.214244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:49.214244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:49.214244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:49.214697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb8999-53df-4b90-a7a1-86159f5d2309 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:49.221192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 389/756] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:49.221192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb62335-9f07-4631-8765-2ea07fcecf3e None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:49.243654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1eeb8999-53df-4b90-a7a1-86159f5d2309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:49.243654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:49.243654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:49.243654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:49.243654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:49.243654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:49.243654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:49.243654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:49.243654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:49.243654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:49.243654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:49.243654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:49.243654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:49.243654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:49.243654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:49.244610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:49.244610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:49.244610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:49.244610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:49.244610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:49.244610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:49.244610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:49.244610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:49.244610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:49.244610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:49.244610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:49.244610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:49.244610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:49.244610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:49.244610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:49.244610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:49.244610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:49.245509 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:49.245509 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:49.245509 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:49.245509 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:49.248140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 368/757] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:49.249142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb8999-53df-4b90-a7a1-86159f5d2309 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:49.336527 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:49.344872 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb62335-9f07-4631-8765-2ea07fcecf3e None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:49.373818 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:49.382680 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb8999-53df-4b90-a7a1-86159f5d2309 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:49.390572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2deafcc-d47b-49fb-8bad-e8fb88d6e543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:49.391581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2deafcc-d47b-49fb-8bad-e8fb88d6e543 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:49.391963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2deafcc-d47b-49fb-8bad-e8fb88d6e543 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:49.392337 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2deafcc-d47b-49fb-8bad-e8fb88d6e543 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:49.400023 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2deafcc-d47b-49fb-8bad-e8fb88d6e543 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:49.401606 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2deafcc-d47b-49fb-8bad-e8fb88d6e543 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:49.418084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-e2deafcc-d47b-49fb-8bad-e8fb88d6e543 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:49.418084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:49.418084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:49.418084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:49.418084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:49.418084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:49.418084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:49.418084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:49.418084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:49.418084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:49.418084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:49.418084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:49.418084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:49.418084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:49.418084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:49.419215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:49.419215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:49.419215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:49.419215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:49.419215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:49.419215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:49.419215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:49.419215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:49.419215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:49.419215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:49.419215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:49.419215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:49.419215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:49.419215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:49.419215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:49.419215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:49.419215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:49.420305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:49.420305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:49.420305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:49.420305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:49.420305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:49.420305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:49.420305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:49.420305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:49.422059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 390/758] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:49.422936 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2deafcc-d47b-49fb-8bad-e8fb88d6e543 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:49.433289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a53ac-84a8-4840-bdeb-99ebcad0d253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:49.434208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a53ac-84a8-4840-bdeb-99ebcad0d253 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:49.434642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a53ac-84a8-4840-bdeb-99ebcad0d253 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:49.435148 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a53ac-84a8-4840-bdeb-99ebcad0d253 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:49.436541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a53ac-84a8-4840-bdeb-99ebcad0d253 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:49.437476 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a53ac-84a8-4840-bdeb-99ebcad0d253 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:49.439678 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88a53ac-84a8-4840-bdeb-99ebcad0d253 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:49.463182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 369/759] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:49.465221 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a53ac-84a8-4840-bdeb-99ebcad0d253 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:49.545481 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2deafcc-d47b-49fb-8bad-e8fb88d6e543 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:49.587592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a511f-4c9e-4d86-b343-1ade4c00fdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:49.588753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a511f-4c9e-4d86-b343-1ade4c00fdc5 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:49.589146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a511f-4c9e-4d86-b343-1ade4c00fdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:49.589573 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a511f-4c9e-4d86-b343-1ade4c00fdc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:49.590566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9a511f-4c9e-4d86-b343-1ade4c00fdc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:49.591327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9a511f-4c9e-4d86-b343-1ade4c00fdc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:49.593095 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9a511f-4c9e-4d86-b343-1ade4c00fdc5 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:49.594113 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a53ac-84a8-4840-bdeb-99ebcad0d253 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:49.606075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 391/760] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:49.607260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a511f-4c9e-4d86-b343-1ade4c00fdc5 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:49.625468 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:49.642444 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189569c-1a54-48bf-8a5e-22c30ca98ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:49.643321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189569c-1a54-48bf-8a5e-22c30ca98ecc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:49.643704 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189569c-1a54-48bf-8a5e-22c30ca98ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:49.644091 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189569c-1a54-48bf-8a5e-22c30ca98ecc None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:49.644992 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189569c-1a54-48bf-8a5e-22c30ca98ecc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:49.645412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189569c-1a54-48bf-8a5e-22c30ca98ecc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:49.647804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189569c-1a54-48bf-8a5e-22c30ca98ecc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:49.661627 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 370/761] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:49.663278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189569c-1a54-48bf-8a5e-22c30ca98ecc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:49.682912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:49.742758 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9a511f-4c9e-4d86-b343-1ade4c00fdc5 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:49.773408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189569c-1a54-48bf-8a5e-22c30ca98ecc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:49.784762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf6b59-b3fb-485c-bd26-1cbe1c070c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:49.785538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf6b59-b3fb-485c-bd26-1cbe1c070c23 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:49.785943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf6b59-b3fb-485c-bd26-1cbe1c070c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:49.786398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf6b59-b3fb-485c-bd26-1cbe1c070c23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:49.787554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf6b59-b3fb-485c-bd26-1cbe1c070c23 None admin] RBAC: Adding query filter params (role.id=2a71a120d6e34544aa5a2031ddf65ec8, scope.project.id=56da0975019c44409b53be473131143a, user.id=df1aeb2be41c49819b811ecfa47d4155) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:49.787983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf6b59-b3fb-485c-bd26-1cbe1c070c23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:49.790106 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf6b59-b3fb-485c-bd26-1cbe1c070c23 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:49.797992 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 392/762] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:23:49 2022] GET /identity/v3/role_assignments?user.id=df1aeb2be41c49819b811ecfa47d4155&scope.project.id=56da0975019c44409b53be473131143a&role.id=2a71a120d6e34544aa5a2031ddf65ec8 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:49.798954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf6b59-b3fb-485c-bd26-1cbe1c070c23 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:49.835726 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2de5f-f21b-4f0f-b838-8a65fb128c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:49.836554 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2de5f-f21b-4f0f-b838-8a65fb128c0b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:49.836850 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2de5f-f21b-4f0f-b838-8a65fb128c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:49.837139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2de5f-f21b-4f0f-b838-8a65fb128c0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:49.845056 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2de5f-f21b-4f0f-b838-8a65fb128c0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:49.846516 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2de5f-f21b-4f0f-b838-8a65fb128c0b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:49.862981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-20c2de5f-f21b-4f0f-b838-8a65fb128c0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:49.862981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:49.862981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:49.862981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:49.862981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:49.862981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:49.862981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:49.862981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:49.862981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:49.862981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:49.862981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:49.862981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:49.862981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:49.862981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:49.862981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:49.863993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:49.863993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:49.863993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:49.863993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:49.863993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:49.863993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:49.863993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:49.863993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:49.863993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:49.863993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:49.863993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:49.863993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:49.863993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:49.863993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:49.863993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:49.863993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:49.863993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:49.864877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:49.864877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:49.864877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:49.864877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:49.864877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:49.864877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:49.864877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:49.864877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:49.870441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 371/763] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:49.872198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610931d-d233-4788-98d2-d86bfbf4cce3 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:49.872495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610931d-d233-4788-98d2-d86bfbf4cce3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:49.872785 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610931d-d233-4788-98d2-d86bfbf4cce3 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:49.874631 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 372/764] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:49.875518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610931d-d233-4788-98d2-d86bfbf4cce3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:49.888580 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:49.915139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf6b59-b3fb-485c-bd26-1cbe1c070c23 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:49.953404 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a85c29-6475-40e1-8074-155364091874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:49.954091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a85c29-6475-40e1-8074-155364091874 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:49.954463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a85c29-6475-40e1-8074-155364091874 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:49.954658 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a85c29-6475-40e1-8074-155364091874 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:49.955538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a85c29-6475-40e1-8074-155364091874 None admin] RBAC: Adding query filter params (role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.project.id=56da0975019c44409b53be473131143a, user.id=8ce35c43e8294f4cbc880f2219a6eb76) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:49.955860 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a85c29-6475-40e1-8074-155364091874 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:49.957539 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a85c29-6475-40e1-8074-155364091874 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:49.971098 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 393/765] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:23:49 2022] GET /identity/v3/role_assignments?user.id=8ce35c43e8294f4cbc880f2219a6eb76&scope.project.id=56da0975019c44409b53be473131143a&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:49.972663 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb8385-e728-4882-a5bc-a9e9a3bb8715 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:49.972960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb8385-e728-4882-a5bc-a9e9a3bb8715 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:49.973246 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb8385-e728-4882-a5bc-a9e9a3bb8715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:49.977017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:50.001709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-cfcb8385-e728-4882-a5bc-a9e9a3bb8715 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:50.003234 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-cfcb8385-e728-4882-a5bc-a9e9a3bb8715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:50.023170 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610931d-d233-4788-98d2-d86bfbf4cce3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:50.083326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 394/766] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:23:50.106898 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb8385-e728-4882-a5bc-a9e9a3bb8715 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:50.120020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acc1ed-c761-4123-b7f8-bc63bf0cec2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:50.120712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acc1ed-c761-4123-b7f8-bc63bf0cec2a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:50.121025 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acc1ed-c761-4123-b7f8-bc63bf0cec2a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:50.121297 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acc1ed-c761-4123-b7f8-bc63bf0cec2a None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:50.122124 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acc1ed-c761-4123-b7f8-bc63bf0cec2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:50.122474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acc1ed-c761-4123-b7f8-bc63bf0cec2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:50.124074 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acc1ed-c761-4123-b7f8-bc63bf0cec2a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:50.152779 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 373/767] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:50.160258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acc1ed-c761-4123-b7f8-bc63bf0cec2a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:50.199672 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:50.238357 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb8385-e728-4882-a5bc-a9e9a3bb8715 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:50.282773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea0c26-017a-4ec7-98fb-49b18890d9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:50.283498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea0c26-017a-4ec7-98fb-49b18890d9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:50.283791 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea0c26-017a-4ec7-98fb-49b18890d9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:50.284554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea0c26-017a-4ec7-98fb-49b18890d9c4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:50.285245 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dea0c26-017a-4ec7-98fb-49b18890d9c4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:50.287832 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dea0c26-017a-4ec7-98fb-49b18890d9c4 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:50.307660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-5dea0c26-017a-4ec7-98fb-49b18890d9c4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 22:23:50.307660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:50.307660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:23:50.307660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:23:50.307660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:23:50.307660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:23:50.307660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:23:50.307660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:23:50.307660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 22:23:50.307660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:50.307660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:23:50.307660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:50.307660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:50.307660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:50.307660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:50.307660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:50.308871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:50.308871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:50.308871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:50.308871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:50.308871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:50.308871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:50.308871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:50.308871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:23:50.308871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:23:50.308871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:23:50.308871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:23:50.308871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:50.308871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:50.308871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:50.308871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:50.308871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:50.309879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:50.309879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:50.309879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:50.309879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:50.309879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:50.309879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:50.309879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:50.309879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:50.309879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:50.309879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:50.309879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:50.309879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:23:50.309879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:23:50.309879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 22:23:50.309879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:50.309879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 395/768] 10.222.0.43 () {56 vars in 1129 bytes} [Thu Jun 23 22:23:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:50.318947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea0c26-017a-4ec7-98fb-49b18890d9c4 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:50.381776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acc1ed-c761-4123-b7f8-bc63bf0cec2a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:50.425367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea0c26-017a-4ec7-98fb-49b18890d9c4 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:50.463834 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9467768-d73e-46fd-a99d-d259cc7517c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:50.464867 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9467768-d73e-46fd-a99d-d259cc7517c1 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:50.465831 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9467768-d73e-46fd-a99d-d259cc7517c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:50.466383 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9467768-d73e-46fd-a99d-d259cc7517c1 None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:50.467707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9467768-d73e-46fd-a99d-d259cc7517c1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:50.468485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9467768-d73e-46fd-a99d-d259cc7517c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:50.470474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9467768-d73e-46fd-a99d-d259cc7517c1 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:50.477511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 396/769] 10.222.0.43 () {56 vars in 1124 bytes} [Thu Jun 23 22:23:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:50.477815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:50.479794 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f971ce0-004b-4168-844e-bf9a327cb9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:50.480627 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f971ce0-004b-4168-844e-bf9a327cb9ab None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:50.480913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f971ce0-004b-4168-844e-bf9a327cb9ab None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:50.481193 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f971ce0-004b-4168-844e-bf9a327cb9ab None admin] PATH_INFO: `/v3/projects/56da0975019c44409b53be473131143a/users/35d686923ef541819de3c77847d38511/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:50.483654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9467768-d73e-46fd-a99d-d259cc7517c1 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:50.491854 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f971ce0-004b-4168-844e-bf9a327cb9ab None admin] RBAC: Authorizing `identity:create_grant(project_id=56da0975019c44409b53be473131143a, user_id=35d686923ef541819de3c77847d38511, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:50.493309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f971ce0-004b-4168-844e-bf9a327cb9ab None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:50.496739 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:50.525456 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 374/770] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:23:50 2022] PUT /identity/v3/projects/56da0975019c44409b53be473131143a/users/35d686923ef541819de3c77847d38511/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:23:50.599328 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9467768-d73e-46fd-a99d-d259cc7517c1 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:50.637837 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624f107-d197-45cb-8d30-be2f878838ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:50.638891 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624f107-d197-45cb-8d30-be2f878838ef None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:50.639478 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624f107-d197-45cb-8d30-be2f878838ef None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:50.639973 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624f107-d197-45cb-8d30-be2f878838ef None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:50.640927 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624f107-d197-45cb-8d30-be2f878838ef None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:50.641449 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624f107-d197-45cb-8d30-be2f878838ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:50.643089 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624f107-d197-45cb-8d30-be2f878838ef None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:50.650285 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 397/771] 10.222.0.43 () {56 vars in 1124 bytes} [Thu Jun 23 22:23:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:50.658959 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f971ce0-004b-4168-844e-bf9a327cb9ab None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:50.669425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ec700-6d2b-4f87-a47d-cde92179d9ce None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:50.670314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ec700-6d2b-4f87-a47d-cde92179d9ce None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:50.670314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ec700-6d2b-4f87-a47d-cde92179d9ce None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:50.672196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 398/772] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:23:50.678120 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4529d5a-40d6-4ac8-bb1b-84a649f49e24 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:50.678562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4529d5a-40d6-4ac8-bb1b-84a649f49e24 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:50.679018 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4529d5a-40d6-4ac8-bb1b-84a649f49e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:50.697639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-c4529d5a-40d6-4ac8-bb1b-84a649f49e24 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:50.699069 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-c4529d5a-40d6-4ac8-bb1b-84a649f49e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:50.764988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:50.775162 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 399/773] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:23:50.801365 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4529d5a-40d6-4ac8-bb1b-84a649f49e24 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:50.853202 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f971ce0-004b-4168-844e-bf9a327cb9ab None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:50.892734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467076e-7a6c-4609-9c2d-9652abdd9e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:50.893412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467076e-7a6c-4609-9c2d-9652abdd9e54 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:50.893706 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467076e-7a6c-4609-9c2d-9652abdd9e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:50.894002 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4467076e-7a6c-4609-9c2d-9652abdd9e54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:50.894880 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467076e-7a6c-4609-9c2d-9652abdd9e54 None admin] RBAC: Adding query filter params (interface=public, service_id=a6297d3bb42a40e4a3994c37b91bd3ef, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:50.895722 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467076e-7a6c-4609-9c2d-9652abdd9e54 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:50.897180 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4467076e-7a6c-4609-9c2d-9652abdd9e54 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:50.907360 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 375/774] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Jun 23 22:23:50 2022] GET /identity/v3/endpoints?service_id=a6297d3bb42a40e4a3994c37b91bd3ef&interface=public®ion_id=RegionOne => generated 194 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:50.913410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467076e-7a6c-4609-9c2d-9652abdd9e54 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:50.933674 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4529d5a-40d6-4ac8-bb1b-84a649f49e24 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:50.994158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157cccf-29c4-4835-8d19-fabbf43ea216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:50.994990 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157cccf-29c4-4835-8d19-fabbf43ea216 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:50.995286 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157cccf-29c4-4835-8d19-fabbf43ea216 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:50.995571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157cccf-29c4-4835-8d19-fabbf43ea216 None admin] PATH_INFO: `/v3/services/network` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:50.996397 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7157cccf-29c4-4835-8d19-fabbf43ea216 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:50.997843 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7157cccf-29c4-4835-8d19-fabbf43ea216 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:51.011199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-7157cccf-29c4-4835-8d19-fabbf43ea216 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 22:23:51.011199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:51.011199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:23:51.011199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:23:51.011199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:23:51.011199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:23:51.011199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:23:51.011199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:23:51.011199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 22:23:51.011199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:51.011199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:23:51.011199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:51.011199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:51.011199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:51.011199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:51.011199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:51.011199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:51.012233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:51.012233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:51.012233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:51.012233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:51.012233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:51.012233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:51.012233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:23:51.012233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:23:51.012233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:23:51.012233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:23:51.012233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:51.012233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:51.012233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:51.012233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:51.012233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:51.012233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:51.013075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:51.013075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:51.013075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:51.013075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:51.013075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:51.013075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:51.013075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:51.013075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:51.013075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:51.013075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:51.013075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:23:51.013075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:23:51.013075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 22:23:51.013075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:51.013075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 400/775] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:50 2022] GET /identity/v3/services/network => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:51.020638 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157cccf-29c4-4835-8d19-fabbf43ea216 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:51.039522 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:51.095643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4467076e-7a6c-4609-9c2d-9652abdd9e54 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:51.134959 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3f16c-c112-4b68-848d-f2155e86df22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:51.135647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3f16c-c112-4b68-848d-f2155e86df22 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:51.135941 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3f16c-c112-4b68-848d-f2155e86df22 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:51.136225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3f16c-c112-4b68-848d-f2155e86df22 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:51.136928 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3f16c-c112-4b68-848d-f2155e86df22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:51.138305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce3f16c-c112-4b68-848d-f2155e86df22 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:51.150988 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 376/776] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Jun 23 22:23:50 2022] GET /identity/v3/services => generated 772 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:51.175888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:51.225482 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157cccf-29c4-4835-8d19-fabbf43ea216 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:51.262667 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372170e-e8c6-4efa-a0f9-f536ba3ec06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:51.263402 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372170e-e8c6-4efa-a0f9-f536ba3ec06b None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:51.263693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372170e-e8c6-4efa-a0f9-f536ba3ec06b None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:51.263980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372170e-e8c6-4efa-a0f9-f536ba3ec06b None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:51.264709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372170e-e8c6-4efa-a0f9-f536ba3ec06b None admin] RBAC: Adding query filter params (name=network) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:51.265025 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372170e-e8c6-4efa-a0f9-f536ba3ec06b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:51.266417 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372170e-e8c6-4efa-a0f9-f536ba3ec06b None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:51.273132 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 401/777] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 22:23:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:51.278579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3f16c-c112-4b68-848d-f2155e86df22 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:51.423832 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3f16c-c112-4b68-848d-f2155e86df22 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:51.449333 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:51.461718 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e173b45-437c-4765-b68e-a822719ff71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:51.462393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e173b45-437c-4765-b68e-a822719ff71f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:51.462687 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e173b45-437c-4765-b68e-a822719ff71f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:51.462996 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e173b45-437c-4765-b68e-a822719ff71f None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:51.463711 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e173b45-437c-4765-b68e-a822719ff71f None admin] RBAC: Adding query filter params (type=network) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:51.464024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e173b45-437c-4765-b68e-a822719ff71f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:51.465396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e173b45-437c-4765-b68e-a822719ff71f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:51.472052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 377/778] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 22:23:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:23:51.477673 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372170e-e8c6-4efa-a0f9-f536ba3ec06b None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:51.589170 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372170e-e8c6-4efa-a0f9-f536ba3ec06b None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:51.612498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:51.625055 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24fcc2-e6cc-4ce8-9938-51448ac5c63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:51.625730 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24fcc2-e6cc-4ce8-9938-51448ac5c63d None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:51.626021 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24fcc2-e6cc-4ce8-9938-51448ac5c63d None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:51.626309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24fcc2-e6cc-4ce8-9938-51448ac5c63d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:51.627138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24fcc2-e6cc-4ce8-9938-51448ac5c63d None admin] RBAC: Adding query filter params (interface=public, service_id=f2d8bccdeb8949d88d3b19ed9fa9dc0b, region_id=RegionOne) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:51.627459 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24fcc2-e6cc-4ce8-9938-51448ac5c63d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:51.628841 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24fcc2-e6cc-4ce8-9938-51448ac5c63d None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:51.637000 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 402/779] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Jun 23 22:23:51 2022] GET /identity/v3/endpoints?service_id=f2d8bccdeb8949d88d3b19ed9fa9dc0b&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:51.643284 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e173b45-437c-4765-b68e-a822719ff71f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:51.766392 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e173b45-437c-4765-b68e-a822719ff71f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:51.802281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d5422-5d69-4104-b7df-4e5653e03d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:51.803762 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d5422-5d69-4104-b7df-4e5653e03d2f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:51.804145 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d5422-5d69-4104-b7df-4e5653e03d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:51.804489 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d5422-5d69-4104-b7df-4e5653e03d2f None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:51.805192 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d5422-5d69-4104-b7df-4e5653e03d2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:51.806574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d5422-5d69-4104-b7df-4e5653e03d2f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:51.812267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 378/780] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Jun 23 22:23:51 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:52.160715 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000b8c7-3cb9-42b4-9ccf-7c3a96842926 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:52.161226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000b8c7-3cb9-42b4-9ccf-7c3a96842926 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:52.161330 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000b8c7-3cb9-42b4-9ccf-7c3a96842926 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:52.163880 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 403/781] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:52.169413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8a29c-b1f0-4c45-b632-8f2baa5d754d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:52.169868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8a29c-b1f0-4c45-b632-8f2baa5d754d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:52.169962 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8a29c-b1f0-4c45-b632-8f2baa5d754d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:52.189729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c6f8a29c-b1f0-4c45-b632-8f2baa5d754d None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:52.190944 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c6f8a29c-b1f0-4c45-b632-8f2baa5d754d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:52.248276 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:52.281435 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 379/782] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:23:52.314352 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000b8c7-3cb9-42b4-9ccf-7c3a96842926 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:52.345098 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14ed4e-40fe-4f41-b969-5eb6d9687a1f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:52.345457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14ed4e-40fe-4f41-b969-5eb6d9687a1f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:52.345649 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14ed4e-40fe-4f41-b969-5eb6d9687a1f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:52.347215 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 380/783] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:23:52.352827 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0486601-d4d9-4cc5-8337-2dcf40eadd08 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:52.353164 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0486601-d4d9-4cc5-8337-2dcf40eadd08 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:52.353445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0486601-d4d9-4cc5-8337-2dcf40eadd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:52.374511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a0486601-d4d9-4cc5-8337-2dcf40eadd08 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:52.375712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a0486601-d4d9-4cc5-8337-2dcf40eadd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:52.504753 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 381/784] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:23:52.512757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000b8c7-3cb9-42b4-9ccf-7c3a96842926 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:52.542736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0486601-d4d9-4cc5-8337-2dcf40eadd08 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:52.560683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e105f9f-d06d-42ba-aad3-47b120bf25b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:52.561626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e105f9f-d06d-42ba-aad3-47b120bf25b6 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:52.561914 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e105f9f-d06d-42ba-aad3-47b120bf25b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:52.562210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e105f9f-d06d-42ba-aad3-47b120bf25b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:52.574021 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e105f9f-d06d-42ba-aad3-47b120bf25b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:52.582207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-5e105f9f-d06d-42ba-aad3-47b120bf25b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:52.582207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:52.582207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:52.582207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:52.582207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:52.582207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:52.582207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:52.582207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:52.582207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:52.582207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:52.582207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:52.582207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:52.582207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:52.582207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:52.582207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:52.582207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:52.583185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:52.583185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:52.583185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:52.583185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:52.583185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:52.583185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:52.583185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:52.583185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:52.583185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:52.583185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:52.583185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:52.583185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:52.583185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:52.583185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:52.583185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:52.583185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:52.584581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:52.584581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:52.584581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:52.584581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:52.584581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:52.584581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:52.584581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:52.584581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:52.584581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:52.584581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:52.584581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:52.584581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:52.584581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 404/785] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:52.597683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e105f9f-d06d-42ba-aad3-47b120bf25b6 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:52.703292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:52.720804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0486601-d4d9-4cc5-8337-2dcf40eadd08 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:52.734533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:52.760527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a121f65-c24e-4f5b-b844-8ca424d2046f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:52.761599 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a121f65-c24e-4f5b-b844-8ca424d2046f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:52.762047 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a121f65-c24e-4f5b-b844-8ca424d2046f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:52.762468 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a121f65-c24e-4f5b-b844-8ca424d2046f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:52.774247 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a121f65-c24e-4f5b-b844-8ca424d2046f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:52.783377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1a121f65-c24e-4f5b-b844-8ca424d2046f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:23:52.783377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:52.783377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:52.783377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:52.783377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:52.783377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:52.783377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:52.783377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:52.783377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:52.783377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:52.783377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:52.783377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:52.783377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:52.783377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:52.783377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:52.783377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:52.784562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:52.784562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:52.784562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:52.784562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:52.784562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:52.784562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:52.784562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:52.784562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:52.784562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:52.784562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:52.784562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:52.784562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:52.784562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:52.784562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:52.784562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:52.784562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:52.785408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:52.785408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:52.785408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:52.785408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:52.785408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:52.785408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:52.785408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:52.785408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:52.785408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:52.785408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:52.785408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:23:52.785408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:52.788678 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 382/786] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:23:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:23:52.792875 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e105f9f-d06d-42ba-aad3-47b120bf25b6 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:52.794239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a121f65-c24e-4f5b-b844-8ca424d2046f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:52.857092 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e6412-5338-4511-992a-94b5cabc8ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:52.857792 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e6412-5338-4511-992a-94b5cabc8ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:52.858238 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e6412-5338-4511-992a-94b5cabc8ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:52.858353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e6412-5338-4511-992a-94b5cabc8ff2 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:52.859212 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e6412-5338-4511-992a-94b5cabc8ff2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:52.859588 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e6412-5338-4511-992a-94b5cabc8ff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:52.860954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e6412-5338-4511-992a-94b5cabc8ff2 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:52.870479 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 405/787] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:23:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:52.879911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e6412-5338-4511-992a-94b5cabc8ff2 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:52.906546 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a121f65-c24e-4f5b-b844-8ca424d2046f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:52.942462 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eabcb0-125a-4bdd-9e74-ba6b9f8b5a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:52.944220 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eabcb0-125a-4bdd-9e74-ba6b9f8b5a45 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:52.944798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eabcb0-125a-4bdd-9e74-ba6b9f8b5a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:52.945314 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eabcb0-125a-4bdd-9e74-ba6b9f8b5a45 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:52.946268 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eabcb0-125a-4bdd-9e74-ba6b9f8b5a45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:52.946770 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eabcb0-125a-4bdd-9e74-ba6b9f8b5a45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:52.948362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eabcb0-125a-4bdd-9e74-ba6b9f8b5a45 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:52.957602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 383/788] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:52.963298 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eabcb0-125a-4bdd-9e74-ba6b9f8b5a45 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:53.012183 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:53.091681 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eabcb0-125a-4bdd-9e74-ba6b9f8b5a45 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:53.137236 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31222be-cd8b-4996-9c53-fb63c6f7d713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:53.138158 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31222be-cd8b-4996-9c53-fb63c6f7d713 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:53.138456 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31222be-cd8b-4996-9c53-fb63c6f7d713 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:53.138742 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31222be-cd8b-4996-9c53-fb63c6f7d713 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:53.141603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e6412-5338-4511-992a-94b5cabc8ff2 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:53.159535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:53.164795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31222be-cd8b-4996-9c53-fb63c6f7d713 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:53.166433 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31222be-cd8b-4996-9c53-fb63c6f7d713 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:53.191050 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f07e4-f296-4339-98a0-eeb98f655090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:53.192507 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f07e4-f296-4339-98a0-eeb98f655090 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:53.193010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f07e4-f296-4339-98a0-eeb98f655090 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:53.193532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f07e4-f296-4339-98a0-eeb98f655090 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:53.209623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f31222be-cd8b-4996-9c53-fb63c6f7d713 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:53.209623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:53.209623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:53.209623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:53.209623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:53.209623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:53.209623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:53.209623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:53.209623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:53.209623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:53.209623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:53.209623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:53.209623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:53.209623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:53.209623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:53.210830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:53.210830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:53.210830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:53.210830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:53.210830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:53.210830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:53.210830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:53.210830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:53.210830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:53.210830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:53.210830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:53.210830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:53.210830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:53.210830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:53.210830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:53.210830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:53.210830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:53.212093 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:53.212093 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:53.212093 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:53.212093 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:53.216125 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 384/789] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:53.217404 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f07e4-f296-4339-98a0-eeb98f655090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:53.219230 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f07e4-f296-4339-98a0-eeb98f655090 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:53.220268 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31222be-cd8b-4996-9c53-fb63c6f7d713 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:53.250536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-df0f07e4-f296-4339-98a0-eeb98f655090 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:53.250536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:53.250536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:53.250536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:53.250536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:53.250536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:53.250536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:53.250536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:53.250536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:53.250536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:53.250536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:53.250536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:53.250536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:53.250536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:53.250536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:53.251639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:53.251639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:53.251639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:53.251639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:53.251639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:53.251639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:53.251639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:53.251639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:53.251639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:53.251639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:53.251639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:53.251639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:53.251639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:53.251639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:53.251639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:53.251639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:53.251639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:53.252526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:53.252526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:53.252526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:53.252526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:53.252526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 406/790] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:53.263974 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f07e4-f296-4339-98a0-eeb98f655090 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:53.330724 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:53.347225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31222be-cd8b-4996-9c53-fb63c6f7d713 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:53.384490 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e990b14-671f-4f58-b0c8-c11d878b965c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:53.385257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e990b14-671f-4f58-b0c8-c11d878b965c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:53.385622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e990b14-671f-4f58-b0c8-c11d878b965c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:53.385994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e990b14-671f-4f58-b0c8-c11d878b965c None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:53.386794 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e990b14-671f-4f58-b0c8-c11d878b965c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:53.387204 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e990b14-671f-4f58-b0c8-c11d878b965c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:53.388644 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e990b14-671f-4f58-b0c8-c11d878b965c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:53.395358 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f07e4-f296-4339-98a0-eeb98f655090 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:53.402181 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 385/791] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:53.403624 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735fd3f-8cf1-482b-87f0-2d4ee91600a7 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:53.403990 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735fd3f-8cf1-482b-87f0-2d4ee91600a7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:53.404349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735fd3f-8cf1-482b-87f0-2d4ee91600a7 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:53.405900 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 386/792] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:53.407916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735fd3f-8cf1-482b-87f0-2d4ee91600a7 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:53.438143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6bc20-376e-4e3e-9df7-364df45fb999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:53.438857 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6bc20-376e-4e3e-9df7-364df45fb999 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:53.439149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6bc20-376e-4e3e-9df7-364df45fb999 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:53.439438 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6bc20-376e-4e3e-9df7-364df45fb999 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:53.440272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6bc20-376e-4e3e-9df7-364df45fb999 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:53.440595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6bc20-376e-4e3e-9df7-364df45fb999 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:53.442084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6bc20-376e-4e3e-9df7-364df45fb999 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:53.460897 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 407/793] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:53.462478 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067b72d-f5e9-49a3-bd7e-9210c7a74cd6 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:53.462780 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067b72d-f5e9-49a3-bd7e-9210c7a74cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:53.463106 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067b72d-f5e9-49a3-bd7e-9210c7a74cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:53.483366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-c067b72d-f5e9-49a3-bd7e-9210c7a74cd6 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:53.484742 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-c067b72d-f5e9-49a3-bd7e-9210c7a74cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:53.501320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:53.516265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735fd3f-8cf1-482b-87f0-2d4ee91600a7 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:53.555202 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76b16c-b034-43e8-8e65-6b0e029be8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:53.556198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76b16c-b034-43e8-8e65-6b0e029be8ef None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:53.556565 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76b16c-b034-43e8-8e65-6b0e029be8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:53.556976 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76b16c-b034-43e8-8e65-6b0e029be8ef None admin] PATH_INFO: `/v3/users/swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:53.564935 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d76b16c-b034-43e8-8e65-6b0e029be8ef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:53.566583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d76b16c-b034-43e8-8e65-6b0e029be8ef None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:53.575164 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 408/794] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:23:53.576232 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067b72d-f5e9-49a3-bd7e-9210c7a74cd6 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:53.583150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5d76b16c-b034-43e8-8e65-6b0e029be8ef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:23:53.583150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:53.583150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:53.583150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:53.583150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:53.583150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:53.583150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:53.583150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:53.583150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:53.583150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:53.583150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:53.583150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:53.583150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:53.583150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:53.583150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:53.583150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:53.584264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:53.584264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:53.584264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:53.584264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:53.584264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:53.584264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:53.584264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:53.584264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:53.584264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:53.584264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:53.584264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:53.584264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:53.584264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:53.584264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:53.584264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:53.584264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:53.584264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:53.584264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:53.585287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:53.585287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:53.585287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:53.585287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:53.585287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:53.585287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:53.585287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:53.585287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:53.585287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:23:53.585287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:53.585287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 387/795] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:23:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:53.592863 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76b16c-b034-43e8-8e65-6b0e029be8ef None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:53.610489 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:53.714278 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067b72d-f5e9-49a3-bd7e-9210c7a74cd6 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:53.715028 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76b16c-b034-43e8-8e65-6b0e029be8ef None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:53.751988 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedad34-2ad2-499d-b495-248f18f6eae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:53.752677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedad34-2ad2-499d-b495-248f18f6eae0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:53.752970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedad34-2ad2-499d-b495-248f18f6eae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:53.753259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedad34-2ad2-499d-b495-248f18f6eae0 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:53.754106 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efedad34-2ad2-499d-b495-248f18f6eae0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:53.754422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efedad34-2ad2-499d-b495-248f18f6eae0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:53.756059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efedad34-2ad2-499d-b495-248f18f6eae0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:53.763945 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29de843-276d-4d36-a41a-b73e6ea8d3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:53.764774 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29de843-276d-4d36-a41a-b73e6ea8d3bc None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:53.765324 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29de843-276d-4d36-a41a-b73e6ea8d3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:53.765612 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29de843-276d-4d36-a41a-b73e6ea8d3bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:53.775102 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 388/796] 10.222.0.43 () {56 vars in 1135 bytes} [Thu Jun 23 22:23:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:53.775883 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedad34-2ad2-499d-b495-248f18f6eae0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:53.779397 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29de843-276d-4d36-a41a-b73e6ea8d3bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:53.780872 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29de843-276d-4d36-a41a-b73e6ea8d3bc None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:53.797700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-d29de843-276d-4d36-a41a-b73e6ea8d3bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:23:53.797700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:53.797700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:53.797700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:53.797700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:53.797700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:53.797700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:53.797700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:53.797700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:53.797700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:53.797700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:53.797700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:53.797700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:53.797700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:53.797700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:53.797700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:53.798707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:53.798707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:53.798707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:53.798707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:53.798707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:53.798707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:53.798707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:53.798707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:53.798707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:53.798707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:53.798707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:53.798707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:53.798707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:53.798707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:53.798707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:53.798707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:53.798707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:53.798707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:53.799678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:53.799678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:53.799678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:53.799678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:53.799678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:53.799678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:53.799678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:53.799678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:53.799678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:23:53.799678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:53.799678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 409/797] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 22:23:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:53.800273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29de843-276d-4d36-a41a-b73e6ea8d3bc None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:53.815365 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:53.861536 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:53.912640 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedad34-2ad2-499d-b495-248f18f6eae0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:53.917284 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29de843-276d-4d36-a41a-b73e6ea8d3bc None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:53.957964 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a41bc-a016-425b-9ada-59f5a4f27813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:53.959454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a41bc-a016-425b-9ada-59f5a4f27813 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:53.959828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a41bc-a016-425b-9ada-59f5a4f27813 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:53.960184 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a41bc-a016-425b-9ada-59f5a4f27813 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:53.964100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eabae6-76ed-43ba-9f54-e2b2c0afd256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:53.965651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eabae6-76ed-43ba-9f54-e2b2c0afd256 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:53.966310 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eabae6-76ed-43ba-9f54-e2b2c0afd256 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:53.966980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eabae6-76ed-43ba-9f54-e2b2c0afd256 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:53.971577 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a41bc-a016-425b-9ada-59f5a4f27813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:53.978765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eabae6-76ed-43ba-9f54-e2b2c0afd256 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:53.979552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-444a41bc-a016-425b-9ada-59f5a4f27813 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:23:53.979552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:53.979552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:53.979552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:53.979552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:53.979552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:53.979552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:53.979552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:53.979552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:53.979552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:53.979552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:53.979552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:53.979552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:53.979552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:53.979552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:53.979552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:53.980467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:53.980467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:53.980467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:53.980467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:53.980467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:53.980467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:53.980467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:53.980467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:53.980467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:53.980467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:53.980467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:53.980467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:53.980467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:53.980467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:53.980467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:53.980467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:53.981304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:53.981304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:53.981304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:53.981304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:53.981304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:53.981304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:53.981304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:53.981304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:53.981304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:53.981304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:53.981304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:23:53.981304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:53.983853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eabae6-76ed-43ba-9f54-e2b2c0afd256 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:53.985792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 389/798] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:23:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:53.986797 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a41bc-a016-425b-9ada-59f5a4f27813 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:54.007059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-57eabae6-76ed-43ba-9f54-e2b2c0afd256 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:54.007059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:54.007059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:54.007059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:54.007059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:54.007059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:54.007059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:54.007059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:54.007059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:54.007059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:54.007059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:54.007059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:54.007059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:54.007059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:54.007059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:54.007998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:54.007998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:54.007998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:54.007998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:54.007998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:54.007998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:54.007998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:54.007998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:54.007998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:54.007998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:54.007998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:54.007998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:54.007998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:54.007998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:54.007998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:54.007998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:54.007998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:54.008886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:54.008886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:54.008886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:54.008886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:54.015529 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 410/799] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:54.016708 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eabae6-76ed-43ba-9f54-e2b2c0afd256 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:54.088884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:54.101213 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a41bc-a016-425b-9ada-59f5a4f27813 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:54.131652 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:54.136139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eabae6-76ed-43ba-9f54-e2b2c0afd256 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:54.143774 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c1055-7fe9-4ff5-b512-1ccccc9bdeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:54.144455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c1055-7fe9-4ff5-b512-1ccccc9bdeb8 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:54.144750 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c1055-7fe9-4ff5-b512-1ccccc9bdeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:54.145034 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c1055-7fe9-4ff5-b512-1ccccc9bdeb8 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:54.146116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c1055-7fe9-4ff5-b512-1ccccc9bdeb8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:54.146225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c1055-7fe9-4ff5-b512-1ccccc9bdeb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:54.148642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c1055-7fe9-4ff5-b512-1ccccc9bdeb8 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:54.171489 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 390/800] 10.222.0.43 () {56 vars in 1149 bytes} [Thu Jun 23 22:23:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:54.172708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c1055-7fe9-4ff5-b512-1ccccc9bdeb8 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:54.177071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b7e66-7c2a-4063-ad8f-1468cdf3060c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:54.177821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b7e66-7c2a-4063-ad8f-1468cdf3060c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:54.178114 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b7e66-7c2a-4063-ad8f-1468cdf3060c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:54.178398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b7e66-7c2a-4063-ad8f-1468cdf3060c None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:54.179184 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b7e66-7c2a-4063-ad8f-1468cdf3060c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:54.179502 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b7e66-7c2a-4063-ad8f-1468cdf3060c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:54.180886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b7e66-7c2a-4063-ad8f-1468cdf3060c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:54.189631 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 411/801] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:54.190981 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740415c-ceb0-417d-afd9-67fd545ecb60 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:54.191296 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740415c-ceb0-417d-afd9-67fd545ecb60 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:54.191817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740415c-ceb0-417d-afd9-67fd545ecb60 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:54.194967 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 412/802] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:54.194967 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740415c-ceb0-417d-afd9-67fd545ecb60 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:54.290457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c1055-7fe9-4ff5-b512-1ccccc9bdeb8 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:54.304753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740415c-ceb0-417d-afd9-67fd545ecb60 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:54.326119 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399839c-4c81-4f2c-b2e6-1726924bdd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:54.326792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399839c-4c81-4f2c-b2e6-1726924bdd3d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:54.327127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399839c-4c81-4f2c-b2e6-1726924bdd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:54.327224 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:54.327404 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399839c-4c81-4f2c-b2e6-1726924bdd3d None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:54.328280 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8399839c-4c81-4f2c-b2e6-1726924bdd3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:54.328544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8399839c-4c81-4f2c-b2e6-1726924bdd3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:54.330066 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8399839c-4c81-4f2c-b2e6-1726924bdd3d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:54.339041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc73657-ed71-49ff-921b-3e63c2fd2c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:54.339777 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc73657-ed71-49ff-921b-3e63c2fd2c71 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:54.340060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc73657-ed71-49ff-921b-3e63c2fd2c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:54.340337 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc73657-ed71-49ff-921b-3e63c2fd2c71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:54.341740 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 391/803] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:54.342538 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399839c-4c81-4f2c-b2e6-1726924bdd3d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:54.351520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc73657-ed71-49ff-921b-3e63c2fd2c71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:54.352954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc73657-ed71-49ff-921b-3e63c2fd2c71 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:54.369817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-8bc73657-ed71-49ff-921b-3e63c2fd2c71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:54.369817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:54.369817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:54.369817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:54.369817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:54.369817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:54.369817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:54.369817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:54.369817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:54.369817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:54.369817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:54.369817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:54.369817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:54.369817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:54.369817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:54.370911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:54.370911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:54.370911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:54.370911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:54.370911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:54.370911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:54.370911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:54.370911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:54.370911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:54.370911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:54.370911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:54.370911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:54.370911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:54.370911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:54.370911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:54.370911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:54.370911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:54.372026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:54.372026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:54.372026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:54.372026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:54.372026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 413/804] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:54.372368 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504f795-9bfd-4272-b5f1-4af7d266b3cd None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:54.372448 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504f795-9bfd-4272-b5f1-4af7d266b3cd None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:54.372709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504f795-9bfd-4272-b5f1-4af7d266b3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:54.388458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-1504f795-9bfd-4272-b5f1-4af7d266b3cd None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:54.389631 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1504f795-9bfd-4272-b5f1-4af7d266b3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:54.391147 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:54.450769 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399839c-4c81-4f2c-b2e6-1726924bdd3d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:54.455270 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 414/805] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:23:54.456572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ad0d6-bbc2-4f80-aea2-af3e57a6356d None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:54.456862 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ad0d6-bbc2-4f80-aea2-af3e57a6356d None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:54.457143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ad0d6-bbc2-4f80-aea2-af3e57a6356d None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:54.463215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 415/806] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:54.463215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ad0d6-bbc2-4f80-aea2-af3e57a6356d None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:54.490720 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deda075-0fd5-4dc8-9d0f-533c9adf230d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:54.491557 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deda075-0fd5-4dc8-9d0f-533c9adf230d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:54.491834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deda075-0fd5-4dc8-9d0f-533c9adf230d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:54.492114 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deda075-0fd5-4dc8-9d0f-533c9adf230d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:54.502506 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deda075-0fd5-4dc8-9d0f-533c9adf230d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:54.504000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deda075-0fd5-4dc8-9d0f-533c9adf230d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:54.521932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-8deda075-0fd5-4dc8-9d0f-533c9adf230d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:54.521932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:54.521932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:54.521932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:54.521932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:54.521932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:54.521932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:54.521932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:54.521932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:54.521932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:54.521932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:54.521932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:54.521932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:54.521932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:54.521932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:54.523208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:54.523208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:54.523208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:54.523208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:54.523208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:54.523208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:54.523208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:54.523208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:54.523208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:54.523208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:54.523208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:54.523208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:54.523208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:54.523208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:54.523208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:54.523208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:54.523208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:54.524122 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:54.524122 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:54.524122 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:54.524122 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:54.524122 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 392/807] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:54.524404 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deda075-0fd5-4dc8-9d0f-533c9adf230d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:54.569883 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ad0d6-bbc2-4f80-aea2-af3e57a6356d None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:54.577852 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:54.604728 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26163efa-06fd-477f-83de-83da71c74c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:54.605452 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26163efa-06fd-477f-83de-83da71c74c41 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:54.605736 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26163efa-06fd-477f-83de-83da71c74c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:54.606016 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26163efa-06fd-477f-83de-83da71c74c41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:54.612538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26163efa-06fd-477f-83de-83da71c74c41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:54.613992 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26163efa-06fd-477f-83de-83da71c74c41 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:54.616199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:54.622891 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deda075-0fd5-4dc8-9d0f-533c9adf230d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:54.627988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-26163efa-06fd-477f-83de-83da71c74c41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:54.627988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:54.627988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:54.627988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:54.627988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:54.627988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:54.627988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:54.627988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:54.627988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:54.627988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:54.627988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:54.627988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:54.627988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:54.627988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:54.627988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:54.628877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:54.628877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:54.628877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:54.628877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:54.628877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:54.628877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:54.628877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:54.628877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:54.628877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:54.628877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:54.628877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:54.628877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:54.628877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:54.628877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:54.628877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:54.628877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:54.628877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:54.629819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:54.629819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:54.629819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:54.629819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:54.629819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:54.629819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:54.629819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:54.629819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:54.629819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 416/808] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:54.630307 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6e0e4-5175-4f56-b746-5bd46288e13e None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:54.631410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6e0e4-5175-4f56-b746-5bd46288e13e None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:54.631508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6e0e4-5175-4f56-b746-5bd46288e13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:54.648302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-e4a6e0e4-5175-4f56-b746-5bd46288e13e None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:54.649496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e4a6e0e4-5175-4f56-b746-5bd46288e13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:54.662407 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab70351-57bb-44c8-9e04-32256843317b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:54.663104 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab70351-57bb-44c8-9e04-32256843317b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:54.663392 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab70351-57bb-44c8-9e04-32256843317b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:54.663678 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab70351-57bb-44c8-9e04-32256843317b None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:54.664396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab70351-57bb-44c8-9e04-32256843317b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:54.664708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab70351-57bb-44c8-9e04-32256843317b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:54.666076 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab70351-57bb-44c8-9e04-32256843317b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:54.677441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 393/809] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:54.678248 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab70351-57bb-44c8-9e04-32256843317b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:54.716543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 417/810] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:23:54.717650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6e0e4-5175-4f56-b746-5bd46288e13e None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:54.788860 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab70351-57bb-44c8-9e04-32256843317b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:54.809997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:54.816033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6e0e4-5175-4f56-b746-5bd46288e13e None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:54.822909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee37f3c-f3e3-4ec0-90d0-430a7f95a2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:54.823747 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee37f3c-f3e3-4ec0-90d0-430a7f95a2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:54.824037 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee37f3c-f3e3-4ec0-90d0-430a7f95a2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:54.824322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee37f3c-f3e3-4ec0-90d0-430a7f95a2d2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:54.824975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee37f3c-f3e3-4ec0-90d0-430a7f95a2d2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:54.826342 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee37f3c-f3e3-4ec0-90d0-430a7f95a2d2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:54.839121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-9ee37f3c-f3e3-4ec0-90d0-430a7f95a2d2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 22:23:54.839121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:54.839121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:23:54.839121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:23:54.839121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:23:54.839121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:23:54.839121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:23:54.839121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:23:54.839121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 22:23:54.839121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:54.839121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:23:54.839121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:54.839121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:54.839121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:54.839121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:54.839121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:54.840070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:54.840070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:54.840070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:54.840070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:54.840070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:54.840070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:54.840070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:54.840070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:23:54.840070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:23:54.840070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:23:54.840070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:23:54.840070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:54.840070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:54.840070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:54.840070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:54.840070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:54.840927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:54.840927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:54.840927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:54.840927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:54.840927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:54.840927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:54.840927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:54.840927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:54.840927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:54.840927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:54.840927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:54.840927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:23:54.840927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:23:54.840927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 22:23:54.840927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:54.840927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 394/811] 10.222.0.43 () {56 vars in 1129 bytes} [Thu Jun 23 22:23:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:54.841833 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee37f3c-f3e3-4ec0-90d0-430a7f95a2d2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:54.855479 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead5a0a-f8f0-4398-a792-fd0cd3ec738e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:54.856141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead5a0a-f8f0-4398-a792-fd0cd3ec738e None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:54.856365 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead5a0a-f8f0-4398-a792-fd0cd3ec738e None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:54.856642 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead5a0a-f8f0-4398-a792-fd0cd3ec738e None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:54.857360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ead5a0a-f8f0-4398-a792-fd0cd3ec738e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:54.857666 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ead5a0a-f8f0-4398-a792-fd0cd3ec738e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:54.859048 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ead5a0a-f8f0-4398-a792-fd0cd3ec738e None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:54.860250 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:54.870251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 418/812] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:54.871179 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead5a0a-f8f0-4398-a792-fd0cd3ec738e None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:54.944405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee37f3c-f3e3-4ec0-90d0-430a7f95a2d2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:54.972807 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead5a0a-f8f0-4398-a792-fd0cd3ec738e None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:54.978477 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbca2d-5438-4041-9816-037c6b5eab3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:54.979174 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbca2d-5438-4041-9816-037c6b5eab3f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:54.979465 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbca2d-5438-4041-9816-037c6b5eab3f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:54.979748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbca2d-5438-4041-9816-037c6b5eab3f None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:54.980541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbbca2d-5438-4041-9816-037c6b5eab3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:54.980857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbbca2d-5438-4041-9816-037c6b5eab3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:54.982283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbbca2d-5438-4041-9816-037c6b5eab3f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:54.994105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 395/813] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:54.994899 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbca2d-5438-4041-9816-037c6b5eab3f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:55.008726 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6fc70-127b-4cde-8bfb-72faa4d138a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:55.009528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6fc70-127b-4cde-8bfb-72faa4d138a7 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:55.009822 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6fc70-127b-4cde-8bfb-72faa4d138a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:55.010110 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6fc70-127b-4cde-8bfb-72faa4d138a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:55.016931 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6fc70-127b-4cde-8bfb-72faa4d138a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:55.018378 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6fc70-127b-4cde-8bfb-72faa4d138a7 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:55.033304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-87e6fc70-127b-4cde-8bfb-72faa4d138a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:55.033304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:55.033304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:55.033304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:55.033304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:55.033304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:55.033304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:55.033304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:55.033304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:55.033304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:55.033304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:55.033304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:55.033304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:55.033304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:55.033304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:55.034371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:55.034371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:55.034371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:55.034371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:55.034371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:55.034371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:55.034371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:55.034371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:55.034371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:55.034371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:55.034371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:55.034371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:55.034371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:55.034371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:55.034371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:55.034371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:55.034371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:55.035454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:55.035454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:55.035454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:55.035454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:55.035454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:55.035454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:55.035454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:55.035454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:55.035454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 419/814] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:55.035454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6fc70-127b-4cde-8bfb-72faa4d138a7 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:55.055346 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:55.076297 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:55.099485 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbca2d-5438-4041-9816-037c6b5eab3f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:55.137558 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642dade3-902d-43e3-b804-fa68cb92b05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:55.138358 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dade3-902d-43e3-b804-fa68cb92b05b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:55.138566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6fc70-127b-4cde-8bfb-72faa4d138a7 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:55.138676 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dade3-902d-43e3-b804-fa68cb92b05b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:55.138906 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dade3-902d-43e3-b804-fa68cb92b05b None admin] PATH_INFO: `/v3/services/network` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:55.139595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642dade3-902d-43e3-b804-fa68cb92b05b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:55.140947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642dade3-902d-43e3-b804-fa68cb92b05b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:55.153290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-642dade3-902d-43e3-b804-fa68cb92b05b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 22:23:55.153290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:55.153290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:23:55.153290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:23:55.153290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:23:55.153290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:23:55.153290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:23:55.153290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:23:55.153290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 22:23:55.153290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:55.153290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:23:55.153290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:55.153290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:55.153290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:55.153290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:55.153290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:55.153290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:55.154405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:55.154405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:55.154405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:55.154405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:55.154405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:55.154405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:55.154405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:23:55.154405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:23:55.154405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:23:55.154405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:23:55.154405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:55.154405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:55.154405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:55.154405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:55.154405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:55.154405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:55.155288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:55.155288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:55.155288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:55.155288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:55.155288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:55.155288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:55.155288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:55.155288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:55.155288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:55.155288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:55.155288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:23:55.155288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:23:55.155288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 22:23:55.155288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:55.155288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 396/815] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:54 2022] GET /identity/v3/services/network => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:55.156088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642dade3-902d-43e3-b804-fa68cb92b05b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:55.178044 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b85d41-7ee3-4db4-9ab0-2d87c9d0a920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:55.178726 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b85d41-7ee3-4db4-9ab0-2d87c9d0a920 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:55.179085 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b85d41-7ee3-4db4-9ab0-2d87c9d0a920 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:55.179373 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b85d41-7ee3-4db4-9ab0-2d87c9d0a920 None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:55.180084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b85d41-7ee3-4db4-9ab0-2d87c9d0a920 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:55.180402 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b85d41-7ee3-4db4-9ab0-2d87c9d0a920 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:55.181779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b85d41-7ee3-4db4-9ab0-2d87c9d0a920 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:55.186231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 420/816] 10.222.0.43 () {56 vars in 1124 bytes} [Thu Jun 23 22:23:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:55.187108 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b85d41-7ee3-4db4-9ab0-2d87c9d0a920 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:55.256769 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642dade3-902d-43e3-b804-fa68cb92b05b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:55.265785 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:55.285196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b85d41-7ee3-4db4-9ab0-2d87c9d0a920 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:55.290623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de47291-a7d4-4f0f-a5be-d32f04fe1870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:55.291448 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de47291-a7d4-4f0f-a5be-d32f04fe1870 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:55.291740 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de47291-a7d4-4f0f-a5be-d32f04fe1870 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:55.292025 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de47291-a7d4-4f0f-a5be-d32f04fe1870 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:55.299364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de47291-a7d4-4f0f-a5be-d32f04fe1870 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:55.300799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de47291-a7d4-4f0f-a5be-d32f04fe1870 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:55.314541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4de47291-a7d4-4f0f-a5be-d32f04fe1870 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:23:55.314541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:55.314541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:55.314541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:55.314541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:55.314541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:55.314541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:55.314541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:55.314541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:55.314541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:55.314541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:55.314541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:55.314541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:55.314541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:55.314541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:55.314541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:55.315667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:55.315667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:55.315667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:55.315667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:55.315667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:55.315667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:55.315667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:55.315667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:55.315667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:55.315667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:55.315667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:55.315667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:55.315667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:55.315667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:55.315667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:55.315667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:55.315667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:55.315667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:55.316744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:55.316744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:55.316744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:55.316744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:55.316744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:55.316744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:55.316744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:55.316744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:55.316744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:23:55.316744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:55.320824 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 397/817] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Jun 23 22:23:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:55.320930 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fcb50-da28-4c12-a254-0d08a110639c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:55.321657 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de47291-a7d4-4f0f-a5be-d32f04fe1870 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:55.321881 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fcb50-da28-4c12-a254-0d08a110639c None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:55.322175 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fcb50-da28-4c12-a254-0d08a110639c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:55.322458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fcb50-da28-4c12-a254-0d08a110639c None admin] PATH_INFO: `/v3/projects/56da0975019c44409b53be473131143a/users/8ce35c43e8294f4cbc880f2219a6eb76/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:55.324316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:55.324471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fcb50-da28-4c12-a254-0d08a110639c None admin] RBAC: Authorizing `identity:create_grant(project_id=56da0975019c44409b53be473131143a, user_id=8ce35c43e8294f4cbc880f2219a6eb76, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:55.326021 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fcb50-da28-4c12-a254-0d08a110639c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:55.339316 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 421/818] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:23:55 2022] PUT /identity/v3/projects/56da0975019c44409b53be473131143a/users/8ce35c43e8294f4cbc880f2219a6eb76/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:23:55.340103 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fcb50-da28-4c12-a254-0d08a110639c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:55.449112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de47291-a7d4-4f0f-a5be-d32f04fe1870 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:55.466023 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fcb50-da28-4c12-a254-0d08a110639c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:55.499209 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f8142-a00c-4393-9506-0e5514aabdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:55.499889 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f8142-a00c-4393-9506-0e5514aabdc7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:55.500182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f8142-a00c-4393-9506-0e5514aabdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:55.500470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f8142-a00c-4393-9506-0e5514aabdc7 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:55.501279 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441f8142-a00c-4393-9506-0e5514aabdc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:55.501601 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441f8142-a00c-4393-9506-0e5514aabdc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:55.503082 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441f8142-a00c-4393-9506-0e5514aabdc7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:55.512685 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404176b2-19af-4534-9cb2-67cc7210a0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:55.513360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404176b2-19af-4534-9cb2-67cc7210a0bf None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:55.513645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404176b2-19af-4534-9cb2-67cc7210a0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:55.513927 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404176b2-19af-4534-9cb2-67cc7210a0bf None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:55.514638 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404176b2-19af-4534-9cb2-67cc7210a0bf None admin] RBAC: Adding query filter params (name=network) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:55.515021 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404176b2-19af-4534-9cb2-67cc7210a0bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:55.516422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 398/819] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:55.516530 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404176b2-19af-4534-9cb2-67cc7210a0bf None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:55.519741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f8142-a00c-4393-9506-0e5514aabdc7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:55.522026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 422/820] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 22:23:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:55.524836 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404176b2-19af-4534-9cb2-67cc7210a0bf None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:55.528209 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:55.599132 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:55.643336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404176b2-19af-4534-9cb2-67cc7210a0bf None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:55.643564 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f8142-a00c-4393-9506-0e5514aabdc7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:55.680611 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9de21f-3da1-47a3-aff8-ce6bb2a81fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:55.681640 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9de21f-3da1-47a3-aff8-ce6bb2a81fbd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:55.682150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9de21f-3da1-47a3-aff8-ce6bb2a81fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:55.682575 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9de21f-3da1-47a3-aff8-ce6bb2a81fbd None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:55.683459 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9de21f-3da1-47a3-aff8-ce6bb2a81fbd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:55.683938 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9de21f-3da1-47a3-aff8-ce6bb2a81fbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:55.685443 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9de21f-3da1-47a3-aff8-ce6bb2a81fbd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:55.686252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993946e-98e4-444b-86ad-623019037d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:55.686973 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993946e-98e4-444b-86ad-623019037d39 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:55.687250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993946e-98e4-444b-86ad-623019037d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:55.687532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993946e-98e4-444b-86ad-623019037d39 None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:55.688367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993946e-98e4-444b-86ad-623019037d39 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:55.688684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993946e-98e4-444b-86ad-623019037d39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:55.690101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e993946e-98e4-444b-86ad-623019037d39 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:55.696294 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 399/821] 10.222.0.43 () {56 vars in 1124 bytes} [Thu Jun 23 22:23:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:55.696294 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9de21f-3da1-47a3-aff8-ce6bb2a81fbd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:55.716918 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 423/822] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Jun 23 22:23:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:55.716918 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993946e-98e4-444b-86ad-623019037d39 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:55.781442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:55.800663 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9de21f-3da1-47a3-aff8-ce6bb2a81fbd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:55.824348 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e993946e-98e4-444b-86ad-623019037d39 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:55.830322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:55.844675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9aaee-fb4c-4069-857f-e5b8457a5c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:55.845678 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9aaee-fb4c-4069-857f-e5b8457a5c19 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:55.845963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9aaee-fb4c-4069-857f-e5b8457a5c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:55.846250 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9aaee-fb4c-4069-857f-e5b8457a5c19 None admin] PATH_INFO: `/v3/projects/56da0975019c44409b53be473131143a/users/df1aeb2be41c49819b811ecfa47d4155/roles/2a71a120d6e34544aa5a2031ddf65ec8` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:55.861994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9aaee-fb4c-4069-857f-e5b8457a5c19 None admin] RBAC: Authorizing `identity:create_grant(project_id=56da0975019c44409b53be473131143a, user_id=df1aeb2be41c49819b811ecfa47d4155, role_id=2a71a120d6e34544aa5a2031ddf65ec8)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:55.863482 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9aaee-fb4c-4069-857f-e5b8457a5c19 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:55.864950 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ff786-19a7-46a6-af73-866357ed1900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:55.866708 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ff786-19a7-46a6-af73-866357ed1900 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:55.867291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ff786-19a7-46a6-af73-866357ed1900 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:55.867401 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ff786-19a7-46a6-af73-866357ed1900 None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:55.868096 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ff786-19a7-46a6-af73-866357ed1900 None admin] RBAC: Adding query filter params (type=network) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:55.868402 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ff786-19a7-46a6-af73-866357ed1900 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:55.869766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ff786-19a7-46a6-af73-866357ed1900 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:55.875159 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 424/823] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 22:23:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:55.875921 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ff786-19a7-46a6-af73-866357ed1900 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:55.890954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 400/824] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:23:55 2022] PUT /identity/v3/projects/56da0975019c44409b53be473131143a/users/df1aeb2be41c49819b811ecfa47d4155/roles/2a71a120d6e34544aa5a2031ddf65ec8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:23:55.890954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9aaee-fb4c-4069-857f-e5b8457a5c19 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:56.006198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9aaee-fb4c-4069-857f-e5b8457a5c19 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:56.026867 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ff786-19a7-46a6-af73-866357ed1900 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:56.044918 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e01317-f592-4d7e-9bdb-ab83c519b23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:56.045576 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e01317-f592-4d7e-9bdb-ab83c519b23b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:56.045863 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e01317-f592-4d7e-9bdb-ab83c519b23b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:56.046143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e01317-f592-4d7e-9bdb-ab83c519b23b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:56.060056 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:56.071566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bec36-53da-447f-8a1c-cbfeae6f92e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:56.072573 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bec36-53da-447f-8a1c-cbfeae6f92e1 None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:56.073099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bec36-53da-447f-8a1c-cbfeae6f92e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:56.073389 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bec36-53da-447f-8a1c-cbfeae6f92e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:56.074784 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741bec36-53da-447f-8a1c-cbfeae6f92e1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:56.075567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741bec36-53da-447f-8a1c-cbfeae6f92e1 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:56.099060 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:56.099963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 425/825] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Jun 23 22:23:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:23:56.101172 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bec36-53da-447f-8a1c-cbfeae6f92e1 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:56.151827 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e01317-f592-4d7e-9bdb-ab83c519b23b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:56.153758 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e01317-f592-4d7e-9bdb-ab83c519b23b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:56.245245 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bec36-53da-447f-8a1c-cbfeae6f92e1 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:56.287882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3ec83-312e-4529-804f-8380e06f0584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:56.289828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3ec83-312e-4529-804f-8380e06f0584 None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:56.290435 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3ec83-312e-4529-804f-8380e06f0584 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:56.290989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3ec83-312e-4529-804f-8380e06f0584 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:56.291897 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3ec83-312e-4529-804f-8380e06f0584 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:56.293445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3ec83-312e-4529-804f-8380e06f0584 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:56.293735 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 401/826] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:55 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:23:56.306962 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e01317-f592-4d7e-9bdb-ab83c519b23b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:56.327575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 426/827] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Jun 23 22:23:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:23:56.410699 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:56.443937 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e01317-f592-4d7e-9bdb-ab83c519b23b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:56.505224 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56226328-2914-408a-b8e1-54863f97d2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:56.507761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56226328-2914-408a-b8e1-54863f97d2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:56.508309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56226328-2914-408a-b8e1-54863f97d2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:56.508792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56226328-2914-408a-b8e1-54863f97d2d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:56.526750 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56226328-2914-408a-b8e1-54863f97d2d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:56.528541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56226328-2914-408a-b8e1-54863f97d2d4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:56.551789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-56226328-2914-408a-b8e1-54863f97d2d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:56.551789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:56.551789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:56.551789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:56.551789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:56.551789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:56.551789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:56.551789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:56.551789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:56.551789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:56.551789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:56.551789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:56.551789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:56.551789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:56.551789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:56.553877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:56.553877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:56.553877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:56.553877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:56.553877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:56.553877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:56.553877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:56.553877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:56.553877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:56.553877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:56.553877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:56.553877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:56.553877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:56.553877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:56.553877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:56.553877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:56.553877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:56.554793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:56.554793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:56.554793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:56.554793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:56.554793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 402/828] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:23:56.559123 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3ec83-312e-4529-804f-8380e06f0584 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:56.607968 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:56.685627 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3ec83-312e-4529-804f-8380e06f0584 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:56.722323 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facca2e6-131f-46a3-ab42-156abe6b9d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:56.727888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facca2e6-131f-46a3-ab42-156abe6b9d07 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:56.727888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facca2e6-131f-46a3-ab42-156abe6b9d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:56.727888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facca2e6-131f-46a3-ab42-156abe6b9d07 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:56.727888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facca2e6-131f-46a3-ab42-156abe6b9d07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:56.727888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facca2e6-131f-46a3-ab42-156abe6b9d07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:56.727888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facca2e6-131f-46a3-ab42-156abe6b9d07 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:56.743008 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 427/829] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:56.748513 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56226328-2914-408a-b8e1-54863f97d2d4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:56.861517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56226328-2914-408a-b8e1-54863f97d2d4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:56.885766 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:56.896613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b7408-c12f-41cd-b166-6d7030b13a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:56.897951 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b7408-c12f-41cd-b166-6d7030b13a75 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:56.898428 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b7408-c12f-41cd-b166-6d7030b13a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:56.898714 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b7408-c12f-41cd-b166-6d7030b13a75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:56.905893 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b7408-c12f-41cd-b166-6d7030b13a75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:56.907373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b7408-c12f-41cd-b166-6d7030b13a75 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:56.923127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-162b7408-c12f-41cd-b166-6d7030b13a75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:56.923127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:56.923127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:56.923127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:56.923127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:56.923127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:56.923127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:56.923127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:56.923127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:56.923127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:56.923127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:56.923127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:56.923127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:56.923127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:56.923127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:56.924131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:56.924131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:56.924131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:56.924131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:56.924131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:56.924131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:56.924131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:56.924131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:56.924131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:56.924131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:56.924131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:56.924131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:56.924131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:56.924131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:56.924131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:56.924131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:56.924131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:56.925011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:56.925011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:56.925011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:56.925011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:56.925011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:56.925011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:56.925011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:56.925011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:56.925011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 403/830] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:56.931848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facca2e6-131f-46a3-ab42-156abe6b9d07 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:57.031920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:57.039094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facca2e6-131f-46a3-ab42-156abe6b9d07 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:57.075401 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db020df-5000-4c07-8c1c-4e2eee2ab044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:57.076079 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db020df-5000-4c07-8c1c-4e2eee2ab044 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:57.076372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db020df-5000-4c07-8c1c-4e2eee2ab044 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:57.076658 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db020df-5000-4c07-8c1c-4e2eee2ab044 None admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:57.077452 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db020df-5000-4c07-8c1c-4e2eee2ab044 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:57.077774 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db020df-5000-4c07-8c1c-4e2eee2ab044 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:57.079238 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db020df-5000-4c07-8c1c-4e2eee2ab044 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:57.092143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 428/831] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:57.098058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b7408-c12f-41cd-b166-6d7030b13a75 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:57.212817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b7408-c12f-41cd-b166-6d7030b13a75 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:57.249058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34ec0e-867a-4d5b-965f-6209e6bce2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:57.250081 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34ec0e-867a-4d5b-965f-6209e6bce2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:57.250573 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34ec0e-867a-4d5b-965f-6209e6bce2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:57.251126 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34ec0e-867a-4d5b-965f-6209e6bce2d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:57.252167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34ec0e-867a-4d5b-965f-6209e6bce2d5 None admin] RBAC: Adding query filter params (role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.project.id=56da0975019c44409b53be473131143a, user.id=35d686923ef541819de3c77847d38511) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:57.252687 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34ec0e-867a-4d5b-965f-6209e6bce2d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:57.254304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34ec0e-867a-4d5b-965f-6209e6bce2d5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:57.261276 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 404/832] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:23:57 2022] GET /identity/v3/role_assignments?user.id=35d686923ef541819de3c77847d38511&scope.project.id=56da0975019c44409b53be473131143a&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:57.497530 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b957bfd-d94e-4b51-96b3-4a6e6dbc1fd0 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:57.498043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b957bfd-d94e-4b51-96b3-4a6e6dbc1fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:57.498123 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b957bfd-d94e-4b51-96b3-4a6e6dbc1fd0 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:57.499873 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 429/833] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:57.505194 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d8d8f-ee10-4575-9d7c-af6b521c4a6e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:57.506272 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d8d8f-ee10-4575-9d7c-af6b521c4a6e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:57.506359 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d8d8f-ee10-4575-9d7c-af6b521c4a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:57.524987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a82d8d8f-ee10-4575-9d7c-af6b521c4a6e None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:57.526210 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a82d8d8f-ee10-4575-9d7c-af6b521c4a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:57.575967 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:57.604292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 405/834] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:23:57.627892 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b957bfd-d94e-4b51-96b3-4a6e6dbc1fd0 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:57.737045 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b957bfd-d94e-4b51-96b3-4a6e6dbc1fd0 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:57.773175 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfeac6-384f-4f5f-a8ba-ccc64b7a3b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:57.773912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfeac6-384f-4f5f-a8ba-ccc64b7a3b7a None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:57.774204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfeac6-384f-4f5f-a8ba-ccc64b7a3b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:57.774490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfeac6-384f-4f5f-a8ba-ccc64b7a3b7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:57.785625 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bfeac6-384f-4f5f-a8ba-ccc64b7a3b7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:57.794151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-e2bfeac6-384f-4f5f-a8ba-ccc64b7a3b7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:23:57.794151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:57.794151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:57.794151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:57.794151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:57.794151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:57.794151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:57.794151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:57.794151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:57.794151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:57.794151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:57.794151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:57.794151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:57.794151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:57.794151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:57.794151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:57.795151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:57.795151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:57.795151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:57.795151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:57.795151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:57.795151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:57.795151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:57.795151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:57.795151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:57.795151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:57.795151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:57.795151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:57.795151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:57.795151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:57.795151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:57.795151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:57.796004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:57.796004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:57.796004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:57.796004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:57.796004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:57.796004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:57.796004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:57.796004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:57.796004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:57.796004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:57.796004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:23:57.796004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:57.796004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 430/835] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:23:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:57.801534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d8d8f-ee10-4575-9d7c-af6b521c4a6e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:57.909199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d8d8f-ee10-4575-9d7c-af6b521c4a6e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:57.945928 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760aa322-ddf3-4f6c-aa82-5ed9f3f9e542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:57.946980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760aa322-ddf3-4f6c-aa82-5ed9f3f9e542 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:57.947502 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760aa322-ddf3-4f6c-aa82-5ed9f3f9e542 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:57.947986 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760aa322-ddf3-4f6c-aa82-5ed9f3f9e542 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:57.948941 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760aa322-ddf3-4f6c-aa82-5ed9f3f9e542 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:57.949454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760aa322-ddf3-4f6c-aa82-5ed9f3f9e542 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:57.951131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760aa322-ddf3-4f6c-aa82-5ed9f3f9e542 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:57.963725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 406/836] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:23:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:57.968977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfeac6-384f-4f5f-a8ba-ccc64b7a3b7a None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:57.985769 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:58.077485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfeac6-384f-4f5f-a8ba-ccc64b7a3b7a None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:58.114959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4deedd0-4e7d-40b9-81a2-019884a5c3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:58.116079 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4deedd0-4e7d-40b9-81a2-019884a5c3db None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:58.116375 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4deedd0-4e7d-40b9-81a2-019884a5c3db None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:58.116663 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4deedd0-4e7d-40b9-81a2-019884a5c3db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:58.128241 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4deedd0-4e7d-40b9-81a2-019884a5c3db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:58.131093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4deedd0-4e7d-40b9-81a2-019884a5c3db None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:58.150534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-d4deedd0-4e7d-40b9-81a2-019884a5c3db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:58.150534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:58.150534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:58.150534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:58.150534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:58.150534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:58.150534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:58.150534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:58.150534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:58.150534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:58.150534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:58.150534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:58.150534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:58.150534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:58.150534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:58.151521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:58.151521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:58.151521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:58.151521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:58.151521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:58.151521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:58.151521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:58.151521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:58.151521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:58.151521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:58.151521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:58.151521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:58.151521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:58.151521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:58.151521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:58.151521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:58.151521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:58.152445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:58.152445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:58.152445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:58.152445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:58.152445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 431/837] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:58.158665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760aa322-ddf3-4f6c-aa82-5ed9f3f9e542 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:58.205830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:58.238957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269526d-b3ff-471d-a0ed-dc0996cf34b3 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:58.238957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269526d-b3ff-471d-a0ed-dc0996cf34b3 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:58.238957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269526d-b3ff-471d-a0ed-dc0996cf34b3 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:58.238957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 432/838] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:23:58.243331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a5cc58-c9bb-4d27-b4c4-83fe68611640 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:58.243628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a5cc58-c9bb-4d27-b4c4-83fe68611640 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:58.243913 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a5cc58-c9bb-4d27-b4c4-83fe68611640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:58.261674 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-e6a5cc58-c9bb-4d27-b4c4-83fe68611640 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:58.262892 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e6a5cc58-c9bb-4d27-b4c4-83fe68611640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:58.270228 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760aa322-ddf3-4f6c-aa82-5ed9f3f9e542 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:58.306776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1dea0-f2dd-4476-9f23-b8fffe2e8ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:58.307664 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1dea0-f2dd-4476-9f23-b8fffe2e8ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:58.308018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1dea0-f2dd-4476-9f23-b8fffe2e8ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:58.308369 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1dea0-f2dd-4476-9f23-b8fffe2e8ad4 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:58.309165 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1dea0-f2dd-4476-9f23-b8fffe2e8ad4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:58.309658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1dea0-f2dd-4476-9f23-b8fffe2e8ad4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:58.311239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1dea0-f2dd-4476-9f23-b8fffe2e8ad4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:58.326170 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 407/839] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:58.333306 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1dea0-f2dd-4476-9f23-b8fffe2e8ad4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:58.339612 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:58.340999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 433/840] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:23:58.361742 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a5cc58-c9bb-4d27-b4c4-83fe68611640 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:58.451686 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1dea0-f2dd-4476-9f23-b8fffe2e8ad4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:58.467239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:58.471531 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a5cc58-c9bb-4d27-b4c4-83fe68611640 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:58.497697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5978d-6180-46db-addc-2a2697aea533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:58.498566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5978d-6180-46db-addc-2a2697aea533 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:58.498801 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5978d-6180-46db-addc-2a2697aea533 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:58.499121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5978d-6180-46db-addc-2a2697aea533 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:58.507220 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5978d-6180-46db-addc-2a2697aea533 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:58.508682 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5978d-6180-46db-addc-2a2697aea533 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:58.512314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df084a-d8a8-45c5-b900-973598e5bad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:58.513123 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df084a-d8a8-45c5-b900-973598e5bad3 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:58.513416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df084a-d8a8-45c5-b900-973598e5bad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:58.513699 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df084a-d8a8-45c5-b900-973598e5bad3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:58.523809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7aa5978d-6180-46db-addc-2a2697aea533 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:23:58.523809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:58.523809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:58.523809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:58.523809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:58.523809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:58.523809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:58.523809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:58.523809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:58.523809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:58.523809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:58.523809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:58.523809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:58.523809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:58.523809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:58.523809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:58.524781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:58.524781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:58.524781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:58.524781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:58.524781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:58.524781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:58.524781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:58.524781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:58.524781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:58.524781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:58.524781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:58.524781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:58.524781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:58.524781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:58.524781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:58.524781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:58.524781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:58.524781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:58.525817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:58.525817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:58.525817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:58.525817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:58.525817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:58.525817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:58.525817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:58.525817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:58.525817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 22:23:58.525817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:58.525817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 408/841] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:23:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:23:58.527705 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df084a-d8a8-45c5-b900-973598e5bad3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:23:58.530960 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc790c8-0f28-40e5-956a-81d4e63cea4b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:58.531247 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc790c8-0f28-40e5-956a-81d4e63cea4b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:58.531531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc790c8-0f28-40e5-956a-81d4e63cea4b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:58.533112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 409/842] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:58.533704 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc790c8-0f28-40e5-956a-81d4e63cea4b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:58.539463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-a4df084a-d8a8-45c5-b900-973598e5bad3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:58.539463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:58.539463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:58.539463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:58.539463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:58.539463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:58.539463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:58.539463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:58.539463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:58.539463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:58.539463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:58.539463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:58.539463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:23:58.539463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:23:58.539463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:23:58.539463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:23:58.540680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:23:58.540680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:23:58.540680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:23:58.540680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:23:58.540680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:58.540680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:58.540680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:58.540680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:58.540680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:58.540680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:58.540680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:58.540680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:58.540680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:58.540680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:58.540680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:58.540680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:58.541564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:58.541564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:58.541564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:58.541564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:58.541564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:23:58.541564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:23:58.541564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:23:58.541564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:23:58.541564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:23:58.541564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:23:58.541564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:23:58.541564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:58.541564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 434/843] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:23:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:23:58.542351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfa3c8-e103-482a-93c6-632a9c72478a None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:58.542649 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfa3c8-e103-482a-93c6-632a9c72478a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:58.542948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bfa3c8-e103-482a-93c6-632a9c72478a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:58.561019 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-c6bfa3c8-e103-482a-93c6-632a9c72478a None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:58.562217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-c6bfa3c8-e103-482a-93c6-632a9c72478a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:58.624036 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:58.646975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 435/844] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:23:58.646975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfa3c8-e103-482a-93c6-632a9c72478a None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:58.657352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc790c8-0f28-40e5-956a-81d4e63cea4b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:58.695242 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676998f-c498-4c76-b819-827acc5e3b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:58.696041 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676998f-c498-4c76-b819-827acc5e3b7c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:58.696337 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676998f-c498-4c76-b819-827acc5e3b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:58.696624 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676998f-c498-4c76-b819-827acc5e3b7c None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:58.697468 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676998f-c498-4c76-b819-827acc5e3b7c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:58.697786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676998f-c498-4c76-b819-827acc5e3b7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:58.699238 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676998f-c498-4c76-b819-827acc5e3b7c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:58.717893 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 410/845] 10.222.0.43 () {56 vars in 1135 bytes} [Thu Jun 23 22:23:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:58.718969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676998f-c498-4c76-b819-827acc5e3b7c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:58.737693 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:58.766083 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bfa3c8-e103-482a-93c6-632a9c72478a None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:58.805563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a8afa-bcaa-4e9d-87f2-a6c66a27e594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:58.806240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a8afa-bcaa-4e9d-87f2-a6c66a27e594 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:58.806536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a8afa-bcaa-4e9d-87f2-a6c66a27e594 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:58.806821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a8afa-bcaa-4e9d-87f2-a6c66a27e594 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:58.807623 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a8afa-bcaa-4e9d-87f2-a6c66a27e594 None admin] RBAC: Adding query filter params (name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:58.807940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a8afa-bcaa-4e9d-87f2-a6c66a27e594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:58.809394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a8afa-bcaa-4e9d-87f2-a6c66a27e594 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:58.817827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 436/846] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:23:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:58.818863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a8afa-bcaa-4e9d-87f2-a6c66a27e594 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:58.828282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676998f-c498-4c76-b819-827acc5e3b7c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:58.867989 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec6dea-f627-4548-87fd-e1c42abaa060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:58.868797 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec6dea-f627-4548-87fd-e1c42abaa060 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:58.869086 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec6dea-f627-4548-87fd-e1c42abaa060 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:58.869375 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec6dea-f627-4548-87fd-e1c42abaa060 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:58.870037 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec6dea-f627-4548-87fd-e1c42abaa060 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:58.871193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:58.871455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec6dea-f627-4548-87fd-e1c42abaa060 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:58.884251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-78ec6dea-f627-4548-87fd-e1c42abaa060 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 22:23:58.884251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:58.884251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:23:58.884251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:23:58.884251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:23:58.884251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:23:58.884251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:23:58.884251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:23:58.884251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 22:23:58.884251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:58.884251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:23:58.884251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:58.884251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:58.884251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:58.884251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:58.884251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:58.884251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:58.890430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:58.890430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:58.890430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:58.890430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:58.890430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:58.890430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:58.890430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:23:58.890430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:23:58.890430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:23:58.890430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:23:58.890430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:58.890430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:58.890430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:58.890430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:58.890430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:58.890430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:58.891553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:58.891553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:58.891553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:58.891553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:58.891553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:58.891553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:58.891553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:58.891553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:58.891553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:58.891553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:58.891553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:23:58.891553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:23:58.891553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 22:23:58.891553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:58.891553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 411/847] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:23:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:58.892527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec6dea-f627-4548-87fd-e1c42abaa060 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:58.933915 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a8afa-bcaa-4e9d-87f2-a6c66a27e594 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:58.971098 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ce6d3-db57-431e-8575-2723dbc33ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:58.972277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ce6d3-db57-431e-8575-2723dbc33ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:58.972564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ce6d3-db57-431e-8575-2723dbc33ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:58.973140 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ce6d3-db57-431e-8575-2723dbc33ae6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:58.977552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:58.999139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec6dea-f627-4548-87fd-e1c42abaa060 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:59.037654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144da2af-0068-4f2b-9f32-25805862f2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:59.038611 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144da2af-0068-4f2b-9f32-25805862f2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:59.039068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144da2af-0068-4f2b-9f32-25805862f2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:59.039359 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144da2af-0068-4f2b-9f32-25805862f2b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:59.041385 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ce6d3-db57-431e-8575-2723dbc33ae6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:59.042830 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ce6d3-db57-431e-8575-2723dbc33ae6 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:59.050461 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144da2af-0068-4f2b-9f32-25805862f2b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:59.051928 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144da2af-0068-4f2b-9f32-25805862f2b6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:59.073982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-144da2af-0068-4f2b-9f32-25805862f2b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:59.073982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:59.073982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:59.073982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:59.073982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:59.073982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:59.073982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:59.073982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:59.073982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:59.073982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:59.073982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:59.073982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:59.073982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:59.073982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:59.073982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:59.074968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:59.074968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:59.074968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:59.074968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:59.074968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:59.074968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:59.074968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:59.074968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:59.074968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:59.074968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:59.074968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:59.074968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:59.074968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:59.074968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:59.074968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:59.074968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:59.074968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:59.075900 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:59.075900 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:59.075900 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:59.075900 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:59.075900 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 412/848] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:59.076789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144da2af-0068-4f2b-9f32-25805862f2b6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:59.098086 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:59.162467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 437/849] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:58 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:23:59.163959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ce6d3-db57-431e-8575-2723dbc33ae6 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:59.195516 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144da2af-0068-4f2b-9f32-25805862f2b6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:59.234473 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db5398-ca5b-4b13-ad4f-717fff8191ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:59.235161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db5398-ca5b-4b13-ad4f-717fff8191ab None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:59.235451 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db5398-ca5b-4b13-ad4f-717fff8191ab None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:59.235727 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db5398-ca5b-4b13-ad4f-717fff8191ab None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:59.236437 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db5398-ca5b-4b13-ad4f-717fff8191ab None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:59.236747 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db5398-ca5b-4b13-ad4f-717fff8191ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:59.238114 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db5398-ca5b-4b13-ad4f-717fff8191ab None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:59.239265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:59.248536 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 413/850] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:23:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:59.248536 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db5398-ca5b-4b13-ad4f-717fff8191ab None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:59.278293 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ce6d3-db57-431e-8575-2723dbc33ae6 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:59.315287 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee765a2c-9318-47e0-b507-f4b4fded1cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:59.316250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee765a2c-9318-47e0-b507-f4b4fded1cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:59.316742 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee765a2c-9318-47e0-b507-f4b4fded1cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:59.317199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee765a2c-9318-47e0-b507-f4b4fded1cb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:59.318179 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee765a2c-9318-47e0-b507-f4b4fded1cb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:59.318688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee765a2c-9318-47e0-b507-f4b4fded1cb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:59.320404 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee765a2c-9318-47e0-b507-f4b4fded1cb1 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:59.332619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 438/851] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:59.333546 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee765a2c-9318-47e0-b507-f4b4fded1cb1 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:59.347439 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:59.351683 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db5398-ca5b-4b13-ad4f-717fff8191ab None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:59.385246 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c6c65-e83c-4593-9aad-69951c140351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:59.386050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c6c65-e83c-4593-9aad-69951c140351 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:59.386340 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c6c65-e83c-4593-9aad-69951c140351 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:59.386628 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c6c65-e83c-4593-9aad-69951c140351 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:59.398090 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c6c65-e83c-4593-9aad-69951c140351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:59.399574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c6c65-e83c-4593-9aad-69951c140351 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:59.416232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-1f5c6c65-e83c-4593-9aad-69951c140351 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:59.416232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:59.416232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:59.416232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:59.416232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:59.416232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:59.416232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:59.416232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:59.416232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:59.416232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:59.416232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:59.416232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:59.416232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:23:59.416232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:23:59.416232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:23:59.417143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:23:59.417143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:59.417143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:59.417143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:59.417143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:59.417143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:59.417143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:59.417143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:59.417143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:59.417143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:59.417143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:59.417143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:59.417143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:59.417143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:59.417143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:59.417143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:59.417143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:59.418052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:23:59.418052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:23:59.418052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:23:59.418052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:59.418052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 414/852] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:23:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:59.418337 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c6c65-e83c-4593-9aad-69951c140351 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:59.448540 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee765a2c-9318-47e0-b507-f4b4fded1cb1 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:59.478875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:59.487915 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36835682-75fe-49fc-b915-e344b4f1beed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:59.488609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36835682-75fe-49fc-b915-e344b4f1beed None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:59.488897 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36835682-75fe-49fc-b915-e344b4f1beed None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:59.489182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36835682-75fe-49fc-b915-e344b4f1beed None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:59.490408 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36835682-75fe-49fc-b915-e344b4f1beed None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:59.490780 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36835682-75fe-49fc-b915-e344b4f1beed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:59.492184 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36835682-75fe-49fc-b915-e344b4f1beed None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:59.496815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 439/853] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:23:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:59.497590 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36835682-75fe-49fc-b915-e344b4f1beed None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:59.538254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c6c65-e83c-4593-9aad-69951c140351 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:59.577868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:59.578418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b55f1-7832-4611-8a2c-fff52ae45a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:59.579045 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b55f1-7832-4611-8a2c-fff52ae45a5b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:59.579340 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b55f1-7832-4611-8a2c-fff52ae45a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:59.579628 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b55f1-7832-4611-8a2c-fff52ae45a5b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:59.588684 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b55f1-7832-4611-8a2c-fff52ae45a5b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:59.590154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b55f1-7832-4611-8a2c-fff52ae45a5b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:59.607302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f79b55f1-7832-4611-8a2c-fff52ae45a5b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:23:59.607302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:59.607302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:59.607302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:59.607302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:59.607302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:59.607302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:59.607302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:59.607302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:59.607302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:59.607302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:59.607302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:59.607302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:23:59.607302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:23:59.607302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:23:59.607302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:23:59.608653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:59.608653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:59.608653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:23:59.608653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:59.608653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:23:59.608653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:23:59.608653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:59.608653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:59.608653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:59.608653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:59.608653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:59.608653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:59.608653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:59.608653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:59.608653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:59.608653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:59.608653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:59.608653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:59.610347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:59.610347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:59.610347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:23:59.610347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:23:59.610347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:23:59.610347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:23:59.610347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:23:59.610347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:23:59.610347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:23:59.610347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:59.614132 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 415/854] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 22:23:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:59.616490 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25743de6-f6d4-4c76-b859-6a3cfed76b46 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:59.616786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25743de6-f6d4-4c76-b859-6a3cfed76b46 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:59.617075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25743de6-f6d4-4c76-b859-6a3cfed76b46 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:59.618988 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 416/855] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:23:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:23:59.619553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25743de6-f6d4-4c76-b859-6a3cfed76b46 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:59.625635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36835682-75fe-49fc-b915-e344b4f1beed None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:59.667419 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ccb9f-c810-4243-b153-4b0f33d39fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:59.668021 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ccb9f-c810-4243-b153-4b0f33d39fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:59.668313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ccb9f-c810-4243-b153-4b0f33d39fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:59.668599 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ccb9f-c810-4243-b153-4b0f33d39fe8 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:59.669325 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ccb9f-c810-4243-b153-4b0f33d39fe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:59.669638 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ccb9f-c810-4243-b153-4b0f33d39fe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:59.671106 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ccb9f-c810-4243-b153-4b0f33d39fe8 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:59.686981 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 440/856] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:59.688439 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc1eca-ef09-4250-a352-bf8e033cfed8 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:59.688735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc1eca-ef09-4250-a352-bf8e033cfed8 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:59.689020 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc1eca-ef09-4250-a352-bf8e033cfed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:59.707071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-d0bc1eca-ef09-4250-a352-bf8e033cfed8 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:23:59.708289 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d0bc1eca-ef09-4250-a352-bf8e033cfed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:23:59.741295 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25743de6-f6d4-4c76-b859-6a3cfed76b46 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:59.753051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:59.787314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 441/857] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:23:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:23:59.787314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc1eca-ef09-4250-a352-bf8e033cfed8 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:59.788555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810e2ce-3827-4d4d-80a8-7d19110b049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:59.789374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810e2ce-3827-4d4d-80a8-7d19110b049b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:59.789667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810e2ce-3827-4d4d-80a8-7d19110b049b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:59.789946 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810e2ce-3827-4d4d-80a8-7d19110b049b None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:59.790849 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6810e2ce-3827-4d4d-80a8-7d19110b049b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:23:59.791199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6810e2ce-3827-4d4d-80a8-7d19110b049b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:59.792626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6810e2ce-3827-4d4d-80a8-7d19110b049b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:59.812771 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 417/858] 10.222.0.43 () {56 vars in 1149 bytes} [Thu Jun 23 22:23:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:23:59.813640 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810e2ce-3827-4d4d-80a8-7d19110b049b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:59.855919 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:23:59.906484 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc1eca-ef09-4250-a352-bf8e033cfed8 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:59.932354 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810e2ce-3827-4d4d-80a8-7d19110b049b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:23:59.941366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca218923-32aa-4f38-9692-f5e790c7c51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:59.942183 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca218923-32aa-4f38-9692-f5e790c7c51c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:59.942469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca218923-32aa-4f38-9692-f5e790c7c51c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:59.942747 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca218923-32aa-4f38-9692-f5e790c7c51c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:59.950601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca218923-32aa-4f38-9692-f5e790c7c51c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:59.952111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca218923-32aa-4f38-9692-f5e790c7c51c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:59.966693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-ca218923-32aa-4f38-9692-f5e790c7c51c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:59.966693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:59.966693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:59.966693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:59.966693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:59.966693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:59.966693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:59.966693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:59.966693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:59.966693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:59.966693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:59.966693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:59.966693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:23:59.966693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:23:59.966693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:23:59.967865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:23:59.967865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:59.967865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:59.967865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:59.967865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:59.967865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:59.967865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:59.967865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:59.967865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:59.967865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:59.967865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:59.967865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:59.967865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:59.967865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:59.967865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:59.967865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:59.967865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:59.970818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:23:59.970818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:23:59.970818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:23:59.970818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:23:59.970818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:23:59.970818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:23:59.970818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:23:59.970818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:23:59.971338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207224db-1ba6-47e3-abc8-5c1522296a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:23:59.971338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207224db-1ba6-47e3-abc8-5c1522296a5d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:23:59.971689 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207224db-1ba6-47e3-abc8-5c1522296a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:23:59.971856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207224db-1ba6-47e3-abc8-5c1522296a5d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:23:59.972540 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207224db-1ba6-47e3-abc8-5c1522296a5d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:23:59.972703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 442/859] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:23:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:59.973511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca218923-32aa-4f38-9692-f5e790c7c51c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:23:59.973916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207224db-1ba6-47e3-abc8-5c1522296a5d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:23:59.987260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-207224db-1ba6-47e3-abc8-5c1522296a5d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 22:23:59.987260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:59.987260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:23:59.987260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:23:59.987260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:23:59.987260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:23:59.987260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:23:59.987260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:23:59.987260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 22:23:59.987260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:59.987260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:23:59.987260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:59.987260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:23:59.987260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:23:59.987260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:23:59.987260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:23:59.988313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:23:59.988313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:23:59.988313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:23:59.988313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:23:59.988313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:23:59.988313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:23:59.988313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:23:59.988313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:23:59.988313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:23:59.988313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:23:59.988313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:23:59.988313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:23:59.988313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:23:59.988313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:23:59.988313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:23:59.988313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:23:59.989163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:23:59.989163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:23:59.989163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:23:59.989163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:23:59.989163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:23:59.989163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:23:59.989163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:23:59.989163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:23:59.989163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:23:59.989163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:23:59.989163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:23:59.989163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:23:59.989163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:23:59.989163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 22:23:59.989163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:23:59.989163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 418/860] 10.222.0.43 () {56 vars in 1132 bytes} [Thu Jun 23 22:23:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:23:59.990490 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207224db-1ba6-47e3-abc8-5c1522296a5d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:00.023488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:00.084959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca218923-32aa-4f38-9692-f5e790c7c51c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:00.096558 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207224db-1ba6-47e3-abc8-5c1522296a5d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:00.108495 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:00.122296 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b247c-8369-420a-91e6-523fb51ea644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:00.123096 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b247c-8369-420a-91e6-523fb51ea644 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:00.123459 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b247c-8369-420a-91e6-523fb51ea644 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:00.123792 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b247c-8369-420a-91e6-523fb51ea644 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:00.141338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd691e-52ba-4d54-af2e-4778076d5c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:00.142030 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd691e-52ba-4d54-af2e-4778076d5c97 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:00.142321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd691e-52ba-4d54-af2e-4778076d5c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:00.142606 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd691e-52ba-4d54-af2e-4778076d5c97 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:00.143465 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd691e-52ba-4d54-af2e-4778076d5c97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:00.143786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd691e-52ba-4d54-af2e-4778076d5c97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:00.145251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd691e-52ba-4d54-af2e-4778076d5c97 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:00.160348 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 419/861] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:23:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:00.161779 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd691e-52ba-4d54-af2e-4778076d5c97 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:00.274026 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:00.284952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd691e-52ba-4d54-af2e-4778076d5c97 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:00.313467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b247c-8369-420a-91e6-523fb51ea644 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:00.315002 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b247c-8369-420a-91e6-523fb51ea644 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:00.323741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ccf4a4-b157-4caa-af6f-f37756771908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:00.324417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccf4a4-b157-4caa-af6f-f37756771908 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:00.324836 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccf4a4-b157-4caa-af6f-f37756771908 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:00.325154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccf4a4-b157-4caa-af6f-f37756771908 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:00.325838 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ccf4a4-b157-4caa-af6f-f37756771908 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:00.326178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ccf4a4-b157-4caa-af6f-f37756771908 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:00.327641 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ccf4a4-b157-4caa-af6f-f37756771908 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:00.336979 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 420/862] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 22:24:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:00.336979 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ccf4a4-b157-4caa-af6f-f37756771908 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:00.423338 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 443/863] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:23:59 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:24:00.423338 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b247c-8369-420a-91e6-523fb51ea644 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:00.443779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:00.446546 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ccf4a4-b157-4caa-af6f-f37756771908 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:00.487308 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8472c47-5a2e-4d28-bd93-e640a4465962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:00.487987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8472c47-5a2e-4d28-bd93-e640a4465962 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:00.488270 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8472c47-5a2e-4d28-bd93-e640a4465962 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:00.488553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8472c47-5a2e-4d28-bd93-e640a4465962 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:00.489512 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8472c47-5a2e-4d28-bd93-e640a4465962 None admin] RBAC: Adding query filter params (role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.project.id=56da0975019c44409b53be473131143a, user.id=8ce35c43e8294f4cbc880f2219a6eb76) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:00.489835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8472c47-5a2e-4d28-bd93-e640a4465962 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:00.491666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8472c47-5a2e-4d28-bd93-e640a4465962 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:00.506956 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 421/864] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:24:00 2022] GET /identity/v3/role_assignments?user.id=8ce35c43e8294f4cbc880f2219a6eb76&scope.project.id=56da0975019c44409b53be473131143a&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:00.506956 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8472c47-5a2e-4d28-bd93-e640a4465962 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:00.534484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:00.535013 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b247c-8369-420a-91e6-523fb51ea644 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:00.578002 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521286cf-1aa0-4ace-aed1-4727c9ade831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:00.578784 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521286cf-1aa0-4ace-aed1-4727c9ade831 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:00.579094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521286cf-1aa0-4ace-aed1-4727c9ade831 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:00.579375 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521286cf-1aa0-4ace-aed1-4727c9ade831 None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:00.580095 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521286cf-1aa0-4ace-aed1-4727c9ade831 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:00.580404 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521286cf-1aa0-4ace-aed1-4727c9ade831 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:00.581791 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521286cf-1aa0-4ace-aed1-4727c9ade831 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:00.588297 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 444/865] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 22:24:00 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:00.634687 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8472c47-5a2e-4d28-bd93-e640a4465962 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:00.676099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5585507-5cf4-4f33-9d93-0b73d19bbf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:00.676935 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5585507-5cf4-4f33-9d93-0b73d19bbf48 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:00.677225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5585507-5cf4-4f33-9d93-0b73d19bbf48 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:00.677569 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5585507-5cf4-4f33-9d93-0b73d19bbf48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:00.692152 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5585507-5cf4-4f33-9d93-0b73d19bbf48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:00.694318 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5585507-5cf4-4f33-9d93-0b73d19bbf48 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:00.714441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f5585507-5cf4-4f33-9d93-0b73d19bbf48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:00.714441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:00.714441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:00.714441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:00.714441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:00.714441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:00.714441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:00.714441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:00.714441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:00.714441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:00.714441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:00.714441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:00.714441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:24:00.714441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:24:00.714441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:24:00.715455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:24:00.715455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:00.715455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:00.715455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:00.715455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:00.715455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:00.715455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:00.715455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:00.715455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:00.715455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:00.715455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:00.715455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:00.715455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:00.715455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:00.715455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:00.715455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:00.715455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:00.716346 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:24:00.716346 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:24:00.716346 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:00.716346 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:00.716346 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 422/866] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:00.722327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521286cf-1aa0-4ace-aed1-4727c9ade831 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:00.817297 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:00.834867 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521286cf-1aa0-4ace-aed1-4727c9ade831 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:00.870304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5a346-a42e-4cd6-9a81-1dd8a80a54a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:00.871064 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5a346-a42e-4cd6-9a81-1dd8a80a54a2 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:00.871358 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5a346-a42e-4cd6-9a81-1dd8a80a54a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:00.871642 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5a346-a42e-4cd6-9a81-1dd8a80a54a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:00.872423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5a346-a42e-4cd6-9a81-1dd8a80a54a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:00.872742 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5a346-a42e-4cd6-9a81-1dd8a80a54a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:00.874132 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5a346-a42e-4cd6-9a81-1dd8a80a54a2 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:00.886192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 445/867] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:00.891543 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5585507-5cf4-4f33-9d93-0b73d19bbf48 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:00.967470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:01.000572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5585507-5cf4-4f33-9d93-0b73d19bbf48 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:01.037744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8412477-3b5c-4329-b44f-961b50caf687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:01.038565 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8412477-3b5c-4329-b44f-961b50caf687 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:01.038855 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8412477-3b5c-4329-b44f-961b50caf687 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:01.039171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8412477-3b5c-4329-b44f-961b50caf687 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:01.046114 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8412477-3b5c-4329-b44f-961b50caf687 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:01.047617 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8412477-3b5c-4329-b44f-961b50caf687 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:01.063382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b8412477-3b5c-4329-b44f-961b50caf687 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:24:01.063382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:01.063382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:01.063382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:01.063382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:01.063382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:01.063382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:01.063382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:01.063382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:01.063382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:01.063382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:01.063382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:01.063382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:24:01.063382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:24:01.063382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:24:01.064316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:24:01.064316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:01.064316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:01.064316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:01.064316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:01.064316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:01.064316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:01.064316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:01.064316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:01.064316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:01.064316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:01.064316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:01.064316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:01.064316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:01.064316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:01.064316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:01.064316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:01.065235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:24:01.065235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:24:01.065235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:24:01.065235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:24:01.065235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:24:01.065235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:24:01.065235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:24:01.065235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:01.065235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 423/868] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:24:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:01.072352 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5a346-a42e-4cd6-9a81-1dd8a80a54a2 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:01.181728 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5a346-a42e-4cd6-9a81-1dd8a80a54a2 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:01.220119 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7bddc4-4335-41be-83a3-e0197b76fe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:01.221177 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bddc4-4335-41be-83a3-e0197b76fe6e None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:01.221737 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bddc4-4335-41be-83a3-e0197b76fe6e None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:01.222221 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bddc4-4335-41be-83a3-e0197b76fe6e None admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:01.223331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7bddc4-4335-41be-83a3-e0197b76fe6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:01.223873 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7bddc4-4335-41be-83a3-e0197b76fe6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:01.226407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7bddc4-4335-41be-83a3-e0197b76fe6e None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:01.243412 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 446/869] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:24:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:01.252012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8412477-3b5c-4329-b44f-961b50caf687 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:01.357593 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8412477-3b5c-4329-b44f-961b50caf687 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:01.392762 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:01.399624 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a645f3-6877-4844-a3e1-5241f5ca022c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:01.400292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a645f3-6877-4844-a3e1-5241f5ca022c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:01.400585 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a645f3-6877-4844-a3e1-5241f5ca022c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:01.400872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a645f3-6877-4844-a3e1-5241f5ca022c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:01.401716 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a645f3-6877-4844-a3e1-5241f5ca022c None admin] RBAC: Adding query filter params (role.id=2a71a120d6e34544aa5a2031ddf65ec8, scope.project.id=56da0975019c44409b53be473131143a, user.id=df1aeb2be41c49819b811ecfa47d4155) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:01.402036 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a645f3-6877-4844-a3e1-5241f5ca022c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:01.403518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a645f3-6877-4844-a3e1-5241f5ca022c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:01.410304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 424/870] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:24:01 2022] GET /identity/v3/role_assignments?user.id=df1aeb2be41c49819b811ecfa47d4155&scope.project.id=56da0975019c44409b53be473131143a&role.id=2a71a120d6e34544aa5a2031ddf65ec8 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:01.599097 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238390b-fd8e-44a2-b601-2c534f8b4ee4 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:01.600224 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238390b-fd8e-44a2-b601-2c534f8b4ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:01.600464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0238390b-fd8e-44a2-b601-2c534f8b4ee4 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:01.601992 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 447/871] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:01.608208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ee07f-b844-43e2-9389-9990c4b3c092 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:01.608443 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ee07f-b844-43e2-9389-9990c4b3c092 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:01.608728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ee07f-b844-43e2-9389-9990c4b3c092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:01.628555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-655ee07f-b844-43e2-9389-9990c4b3c092 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:01.630142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-655ee07f-b844-43e2-9389-9990c4b3c092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:01.702766 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 425/872] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:01.726810 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238390b-fd8e-44a2-b601-2c534f8b4ee4 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:01.736834 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:01.839143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0238390b-fd8e-44a2-b601-2c534f8b4ee4 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:01.876986 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cfc9e-0747-4e8c-a76d-267b4f69824f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:01.877582 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cfc9e-0747-4e8c-a76d-267b4f69824f None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:01.877874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cfc9e-0747-4e8c-a76d-267b4f69824f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:01.878158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cfc9e-0747-4e8c-a76d-267b4f69824f None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:01.878896 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5cfc9e-0747-4e8c-a76d-267b4f69824f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:01.880292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5cfc9e-0747-4e8c-a76d-267b4f69824f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:01.898261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 448/873] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 22:24:01 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:24:02.770911 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2757e-e907-4773-bc91-1c69e7fcf9f1 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:02.772031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2757e-e907-4773-bc91-1c69e7fcf9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:02.772369 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2757e-e907-4773-bc91-1c69e7fcf9f1 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:02.774364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 426/874] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:02.779472 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8681503-1517-4940-91cc-c81bc56ad4c8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:02.779616 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75b01f-a177-4d4f-994a-d4ed298cbe32 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:02.780002 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8681503-1517-4940-91cc-c81bc56ad4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:02.780111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75b01f-a177-4d4f-994a-d4ed298cbe32 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:02.780209 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75b01f-a177-4d4f-994a-d4ed298cbe32 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:02.780465 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8681503-1517-4940-91cc-c81bc56ad4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:02.781727 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 449/875] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:02.786420 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de1403-a1fe-4f24-9d37-eafb35fe8215 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:02.786713 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de1403-a1fe-4f24-9d37-eafb35fe8215 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:02.787585 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de1403-a1fe-4f24-9d37-eafb35fe8215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:02.804327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c8681503-1517-4940-91cc-c81bc56ad4c8 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:02.805240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-41de1403-a1fe-4f24-9d37-eafb35fe8215 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:02.806055 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c8681503-1517-4940-91cc-c81bc56ad4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:02.806387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-41de1403-a1fe-4f24-9d37-eafb35fe8215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:02.878403 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:02.906944 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 427/877] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:02 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:02.907127 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 450/877] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:02 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:02.920581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8681503-1517-4940-91cc-c81bc56ad4c8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:02.924811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de1403-a1fe-4f24-9d37-eafb35fe8215 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:03.003294 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:03.029572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8681503-1517-4940-91cc-c81bc56ad4c8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:03.032986 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de1403-a1fe-4f24-9d37-eafb35fe8215 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:03.067592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc698412-1c96-4150-b62e-d9e36137ee78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:03.069219 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc698412-1c96-4150-b62e-d9e36137ee78 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:03.069715 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc698412-1c96-4150-b62e-d9e36137ee78 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:03.070186 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc698412-1c96-4150-b62e-d9e36137ee78 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:03.070518 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c370e-d249-47e8-a375-c6041b6d6a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:03.071477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c370e-d249-47e8-a375-c6041b6d6a3c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:03.071817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc698412-1c96-4150-b62e-d9e36137ee78 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:03.071922 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c370e-d249-47e8-a375-c6041b6d6a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:03.072172 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c370e-d249-47e8-a375-c6041b6d6a3c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:03.073008 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c370e-d249-47e8-a375-c6041b6d6a3c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:03.073961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc698412-1c96-4150-b62e-d9e36137ee78 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:03.074520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c370e-d249-47e8-a375-c6041b6d6a3c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:03.088940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-c10c370e-d249-47e8-a375-c6041b6d6a3c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:24:03.088940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:03.088940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:24:03.088940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:24:03.088940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:24:03.088940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:24:03.088940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:24:03.088940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:03.088940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:24:03.088940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:03.088940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:24:03.088940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:03.088940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:03.088940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:03.088940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:03.088940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:03.088940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:03.090058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:03.090058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:03.090058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:03.090058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:03.090058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:03.090058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:03.090058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:24:03.090058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:24:03.090058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:24:03.090058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:24:03.090058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:03.090058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:03.090058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:03.090058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:03.090058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:03.090058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:03.090954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:03.090954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:03.090954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:03.090954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:03.090954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:03.090954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:03.090954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:03.090954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:03.090954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:03.090954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:03.090954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:24:03.090954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:03.090954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:24:03.090954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:03.095335 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 451/878] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 22:24:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:03.096338 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c370e-d249-47e8-a375-c6041b6d6a3c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:03.102982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 428/879] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 22:24:02 2022] POST /identity/v3/services => generated 269 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:24:03.180419 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:03.229416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c370e-d249-47e8-a375-c6041b6d6a3c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:03.273888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9638ff-ee3b-4eb3-a496-3ede889f5f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:03.274570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9638ff-ee3b-4eb3-a496-3ede889f5f51 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:03.274859 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9638ff-ee3b-4eb3-a496-3ede889f5f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:03.275174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9638ff-ee3b-4eb3-a496-3ede889f5f51 None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:03.275883 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9638ff-ee3b-4eb3-a496-3ede889f5f51 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:03.276196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9638ff-ee3b-4eb3-a496-3ede889f5f51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:03.277583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9638ff-ee3b-4eb3-a496-3ede889f5f51 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:03.283243 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 452/880] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 22:24:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:03.288910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc698412-1c96-4150-b62e-d9e36137ee78 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:03.393648 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc698412-1c96-4150-b62e-d9e36137ee78 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:03.411014 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:03.430880 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3fc16-23b4-4c5e-b5ff-fb4e76d202ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:03.431791 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3fc16-23b4-4c5e-b5ff-fb4e76d202ad None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:03.432149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3fc16-23b4-4c5e-b5ff-fb4e76d202ad None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:03.432673 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3fc16-23b4-4c5e-b5ff-fb4e76d202ad None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:03.433393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3fc16-23b4-4c5e-b5ff-fb4e76d202ad None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:03.433708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3fc16-23b4-4c5e-b5ff-fb4e76d202ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:03.435110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c3fc16-23b4-4c5e-b5ff-fb4e76d202ad None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:03.440117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 429/881] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 22:24:03 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:03.679753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120df94d-e790-4dc3-be3a-3eb034d7fb5a None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:03.680090 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120df94d-e790-4dc3-be3a-3eb034d7fb5a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:03.680288 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120df94d-e790-4dc3-be3a-3eb034d7fb5a None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:03.681877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 453/882] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:03.687120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0688bb-d027-478d-af4f-4b8c2c8fb432 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:03.687421 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0688bb-d027-478d-af4f-4b8c2c8fb432 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:03.687710 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0688bb-d027-478d-af4f-4b8c2c8fb432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:03.705716 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8f0688bb-d027-478d-af4f-4b8c2c8fb432 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:03.707287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8f0688bb-d027-478d-af4f-4b8c2c8fb432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:03.777909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 430/883] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:03.801704 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120df94d-e790-4dc3-be3a-3eb034d7fb5a None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:03.912235 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120df94d-e790-4dc3-be3a-3eb034d7fb5a None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:03.930263 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:03.950515 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4be7b-baad-405b-9d63-be9228b60408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:03.951491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4be7b-baad-405b-9d63-be9228b60408 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:03.951781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4be7b-baad-405b-9d63-be9228b60408 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:03.952070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4be7b-baad-405b-9d63-be9228b60408 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:03.963599 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4be7b-baad-405b-9d63-be9228b60408 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:24:03.965238 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a9ddd-5c7a-409f-8097-cc3ab02782d0 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:03.965538 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a9ddd-5c7a-409f-8097-cc3ab02782d0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:03.965830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a9ddd-5c7a-409f-8097-cc3ab02782d0 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:03.967446 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 431/884] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:03.971874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-a8e4be7b-baad-405b-9d63-be9228b60408 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:24:03.971874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:03.971874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:03.971874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:03.971874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:03.971874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:03.971874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:03.971874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:03.971874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:03.971874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:03.971874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:03.971874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:03.971874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:24:03.971874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:24:03.971874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:24:03.971874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:24:03.972828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:24:03.972828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:24:03.972828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:24:03.972828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:24:03.972828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:03.972828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:03.972828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:03.972828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:03.972828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:03.972828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:03.972828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:03.972828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:03.972828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:03.972828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:03.972828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:03.972828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:03.973695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:03.973695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:03.973695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:03.973695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:03.973695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:24:03.973695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:24:03.973695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:24:03.973695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:24:03.973695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:24:03.973695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:24:03.973695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:24:03.973695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:03.973695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 454/885] 10.222.0.43 () {56 vars in 1120 bytes} [Thu Jun 23 22:24:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:03.975568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d00c0-e732-48bd-83ca-1ff84ce887e6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:03.975865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d00c0-e732-48bd-83ca-1ff84ce887e6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:03.976150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d00c0-e732-48bd-83ca-1ff84ce887e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:03.978690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4be7b-baad-405b-9d63-be9228b60408 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:03.993919 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-143d00c0-e732-48bd-83ca-1ff84ce887e6 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:03.995168 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-143d00c0-e732-48bd-83ca-1ff84ce887e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:04.070066 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 432/886] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:04.093724 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d00c0-e732-48bd-83ca-1ff84ce887e6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:04.103718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4be7b-baad-405b-9d63-be9228b60408 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:04.148074 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:04.151863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c60326-b35d-41c1-979c-d5b804dfecca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:04.152553 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c60326-b35d-41c1-979c-d5b804dfecca None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:04.152963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c60326-b35d-41c1-979c-d5b804dfecca None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:04.153255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c60326-b35d-41c1-979c-d5b804dfecca None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:04.154514 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c60326-b35d-41c1-979c-d5b804dfecca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:04.155473 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c60326-b35d-41c1-979c-d5b804dfecca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:04.157101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c60326-b35d-41c1-979c-d5b804dfecca None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:04.169084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 455/887] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Jun 23 22:24:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:04.173790 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c60326-b35d-41c1-979c-d5b804dfecca None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:04.217985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d00c0-e732-48bd-83ca-1ff84ce887e6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:04.233121 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:04.257067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba4aba-9c06-4567-ac73-17d4c15e90ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:04.258774 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba4aba-9c06-4567-ac73-17d4c15e90ac None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:04.259386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba4aba-9c06-4567-ac73-17d4c15e90ac None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:04.259972 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba4aba-9c06-4567-ac73-17d4c15e90ac None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:04.260836 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba4aba-9c06-4567-ac73-17d4c15e90ac None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:04.262497 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba4aba-9c06-4567-ac73-17d4c15e90ac None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:04.277809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-61ba4aba-9c06-4567-ac73-17d4c15e90ac None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 22:24:04.277809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:04.277809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:24:04.277809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:24:04.277809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:24:04.277809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:24:04.277809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:24:04.277809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:04.277809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 22:24:04.277809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:04.277809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:24:04.277809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:04.277809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:04.277809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:04.277809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:04.277809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:04.277809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:04.278969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:04.278969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:04.278969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:04.278969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:04.278969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:04.278969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:04.278969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:24:04.278969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:24:04.278969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:24:04.278969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:24:04.278969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:04.278969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:04.278969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:04.278969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:04.278969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:04.278969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:04.279828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:04.279828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:04.279828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:04.279828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:04.279828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:04.279828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:04.279828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:04.279828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:04.279828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:04.279828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:04.279828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:24:04.279828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:04.279828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 22:24:04.279828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:04.287805 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 433/888] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:24:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:04.289541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba4aba-9c06-4567-ac73-17d4c15e90ac None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:04.306747 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c60326-b35d-41c1-979c-d5b804dfecca None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:04.357489 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e1d8c-020a-4905-959c-8e505aeab228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:04.358279 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e1d8c-020a-4905-959c-8e505aeab228 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:04.358577 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e1d8c-020a-4905-959c-8e505aeab228 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:04.358855 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e1d8c-020a-4905-959c-8e505aeab228 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:04.374653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e1d8c-020a-4905-959c-8e505aeab228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:04.376176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e1d8c-020a-4905-959c-8e505aeab228 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:04.400101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-904e1d8c-020a-4905-959c-8e505aeab228 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:04.400101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:04.400101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:04.400101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:04.400101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:04.400101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:04.400101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:04.400101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:04.400101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:04.400101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:04.400101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:04.400101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:04.400101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:24:04.400101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:24:04.400101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:24:04.401041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:24:04.401041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:04.401041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:04.401041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:04.401041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:04.401041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:04.401041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:04.401041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:04.401041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:04.401041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:04.401041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:04.401041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:04.401041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:04.401041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:04.401041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:04.401041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:04.401041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:04.401948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:24:04.401948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:24:04.401948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:04.401948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:04.401948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 456/889] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:04.407853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e1d8c-020a-4905-959c-8e505aeab228 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:04.409480 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba4aba-9c06-4567-ac73-17d4c15e90ac None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:04.412436 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:04.448880 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e492db-275e-4fe1-85eb-ee9efd85919b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:04.449651 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e492db-275e-4fe1-85eb-ee9efd85919b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:04.449940 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e492db-275e-4fe1-85eb-ee9efd85919b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:04.450218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e492db-275e-4fe1-85eb-ee9efd85919b None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:04.451000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e492db-275e-4fe1-85eb-ee9efd85919b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:04.451311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e492db-275e-4fe1-85eb-ee9efd85919b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:04.452730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e492db-275e-4fe1-85eb-ee9efd85919b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:04.462578 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 434/890] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:24:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:04.463560 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e492db-275e-4fe1-85eb-ee9efd85919b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:04.523683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:04.534171 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e1d8c-020a-4905-959c-8e505aeab228 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:04.572993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1174392-769a-4756-8a08-a01f2482202c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:04.573568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e492db-275e-4fe1-85eb-ee9efd85919b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:04.573903 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1174392-769a-4756-8a08-a01f2482202c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:04.574292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1174392-769a-4756-8a08-a01f2482202c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:04.574678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1174392-769a-4756-8a08-a01f2482202c None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:04.575648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1174392-769a-4756-8a08-a01f2482202c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:04.576120 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1174392-769a-4756-8a08-a01f2482202c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:04.577583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1174392-769a-4756-8a08-a01f2482202c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:04.591137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 457/891] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:04.596700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1174392-769a-4756-8a08-a01f2482202c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:04.611188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4ab42-9c43-442d-a999-1d15ada0402b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:04.611918 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4ab42-9c43-442d-a999-1d15ada0402b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:04.612205 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4ab42-9c43-442d-a999-1d15ada0402b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:04.612487 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4ab42-9c43-442d-a999-1d15ada0402b None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:04.613262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4ab42-9c43-442d-a999-1d15ada0402b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:04.613628 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4ab42-9c43-442d-a999-1d15ada0402b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:04.615709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4ab42-9c43-442d-a999-1d15ada0402b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:04.622030 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 435/892] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:24:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:04.629798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4ab42-9c43-442d-a999-1d15ada0402b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:04.673482 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:04.719455 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1174392-769a-4756-8a08-a01f2482202c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:04.759221 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f574e7a-c3d1-44e6-89f0-2b64aebb20ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:04.760332 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f574e7a-c3d1-44e6-89f0-2b64aebb20ea None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:04.760774 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f574e7a-c3d1-44e6-89f0-2b64aebb20ea None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:04.761254 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f574e7a-c3d1-44e6-89f0-2b64aebb20ea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:04.763142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4ab42-9c43-442d-a999-1d15ada0402b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:04.770667 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f574e7a-c3d1-44e6-89f0-2b64aebb20ea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:04.773302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f574e7a-c3d1-44e6-89f0-2b64aebb20ea None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:04.789350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-4f574e7a-c3d1-44e6-89f0-2b64aebb20ea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:24:04.789350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:04.789350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:04.789350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:04.789350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:04.789350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:04.789350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:04.789350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:04.789350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:04.789350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:04.789350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:04.789350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:04.789350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:24:04.789350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:24:04.789350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:24:04.789350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:24:04.790333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:04.790333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:04.790333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:24:04.790333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:24:04.790333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:24:04.790333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:24:04.790333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:04.790333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:04.790333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:04.790333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:04.790333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:04.790333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:04.790333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:04.790333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:04.790333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:04.790333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:04.790333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:04.790333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:04.791311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:04.791311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:04.791311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:24:04.791311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:24:04.791311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:24:04.791311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:24:04.791311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:24:04.791311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:24:04.791311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:24:04.791311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:04.793918 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 458/893] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 22:24:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:04.801118 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f574e7a-c3d1-44e6-89f0-2b64aebb20ea None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:04.807947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe977bdd-7b06-40bf-a6ab-e66bdbece3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:04.808623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe977bdd-7b06-40bf-a6ab-e66bdbece3ba None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:04.808912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe977bdd-7b06-40bf-a6ab-e66bdbece3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:04.809197 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe977bdd-7b06-40bf-a6ab-e66bdbece3ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:04.810002 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe977bdd-7b06-40bf-a6ab-e66bdbece3ba None admin] RBAC: Adding query filter params (interface=public, service_id=23010975420e48b2a2b132a70dc15f7a, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:04.810322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe977bdd-7b06-40bf-a6ab-e66bdbece3ba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:04.811776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe977bdd-7b06-40bf-a6ab-e66bdbece3ba None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:04.817760 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:04.822691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 436/894] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Jun 23 22:24:04 2022] GET /identity/v3/endpoints?service_id=23010975420e48b2a2b132a70dc15f7a&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:04.824154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe977bdd-7b06-40bf-a6ab-e66bdbece3ba None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:04.946646 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f574e7a-c3d1-44e6-89f0-2b64aebb20ea None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:04.957853 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:04.968542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe977bdd-7b06-40bf-a6ab-e66bdbece3ba None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:04.997558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f525c1-48f7-4cf2-819d-e9e28b97a971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:04.999203 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f525c1-48f7-4cf2-819d-e9e28b97a971 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:05.000010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f525c1-48f7-4cf2-819d-e9e28b97a971 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:05.000502 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f525c1-48f7-4cf2-819d-e9e28b97a971 None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:05.002219 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f525c1-48f7-4cf2-819d-e9e28b97a971 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:05.003041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f525c1-48f7-4cf2-819d-e9e28b97a971 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:05.005997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f525c1-48f7-4cf2-819d-e9e28b97a971 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:05.011510 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689a261-39cb-406c-9c4a-839edb0e5a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:05.012191 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689a261-39cb-406c-9c4a-839edb0e5a7e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:05.012485 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689a261-39cb-406c-9c4a-839edb0e5a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:05.012780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689a261-39cb-406c-9c4a-839edb0e5a7e None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:05.013473 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5689a261-39cb-406c-9c4a-839edb0e5a7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:05.015094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5689a261-39cb-406c-9c4a-839edb0e5a7e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:05.027140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 437/895] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Jun 23 22:24:04 2022] GET /identity/v3/services => generated 1268 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:24:05.041674 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 459/896] 10.222.0.43 () {56 vars in 1149 bytes} [Thu Jun 23 22:24:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:05.043584 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689a261-39cb-406c-9c4a-839edb0e5a7e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:05.165818 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689a261-39cb-406c-9c4a-839edb0e5a7e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:05.204549 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e40cde-e1ae-4f4e-bacb-ebdfe0401b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:05.205208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e40cde-e1ae-4f4e-bacb-ebdfe0401b2c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:05.205497 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e40cde-e1ae-4f4e-bacb-ebdfe0401b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:05.205777 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e40cde-e1ae-4f4e-bacb-ebdfe0401b2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:05.238061 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:05.283474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d4dd1-1c9c-4739-bbe1-003f583de1d0 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:05.283767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d4dd1-1c9c-4739-bbe1-003f583de1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:05.284046 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d4dd1-1c9c-4739-bbe1-003f583de1d0 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:05.285618 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 460/897] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:05.286041 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e40cde-e1ae-4f4e-bacb-ebdfe0401b2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:05.287602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e40cde-e1ae-4f4e-bacb-ebdfe0401b2c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:05.291066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d0175-3f04-41f7-a95b-3e7f04988e51 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:05.291341 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d0175-3f04-41f7-a95b-3e7f04988e51 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:05.291628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d0175-3f04-41f7-a95b-3e7f04988e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:05.313310 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-1d1d0175-3f04-41f7-a95b-3e7f04988e51 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:05.315492 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1d1d0175-3f04-41f7-a95b-3e7f04988e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:05.405393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:05.411153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 438/898] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:24:05 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:24:05.416852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e40cde-e1ae-4f4e-bacb-ebdfe0401b2c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:05.423397 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 461/899] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:05.448790 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d0175-3f04-41f7-a95b-3e7f04988e51 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:05.485821 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:05.533255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e40cde-e1ae-4f4e-bacb-ebdfe0401b2c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:05.558275 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d0175-3f04-41f7-a95b-3e7f04988e51 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:05.573564 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887439c-2577-49cf-b88f-3d834f3d3241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:05.574767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887439c-2577-49cf-b88f-3d834f3d3241 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:05.575321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887439c-2577-49cf-b88f-3d834f3d3241 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:05.575804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887439c-2577-49cf-b88f-3d834f3d3241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:05.588279 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887439c-2577-49cf-b88f-3d834f3d3241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:05.590243 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b887439c-2577-49cf-b88f-3d834f3d3241 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:05.595537 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf97239-5b1d-489d-b8fb-4cd1528abe72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:05.596362 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf97239-5b1d-489d-b8fb-4cd1528abe72 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:05.596651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf97239-5b1d-489d-b8fb-4cd1528abe72 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:05.596932 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf97239-5b1d-489d-b8fb-4cd1528abe72 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:05.597594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf97239-5b1d-489d-b8fb-4cd1528abe72 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:05.598973 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf97239-5b1d-489d-b8fb-4cd1528abe72 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:05.612374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b887439c-2577-49cf-b88f-3d834f3d3241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:05.612374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:05.612374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:05.612374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:05.612374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:05.612374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:05.612374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:05.612374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:05.612374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:05.612374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:05.612374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:05.612374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:05.612374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:24:05.612374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:24:05.612374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:24:05.613281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:24:05.613281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:05.613281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:05.613281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:05.613281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:05.613281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:05.613281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:05.613281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:05.613281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:05.613281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:05.613281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:05.613281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:05.613281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:05.613281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:05.613281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:05.613281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:05.613281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:05.614173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:24:05.614173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:24:05.614173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:05.614173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:05.614391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-cbf97239-5b1d-489d-b8fb-4cd1528abe72 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 22:24:05.614391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:05.614391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:24:05.614391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:24:05.614391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:24:05.614391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:24:05.614391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:24:05.614391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:05.614391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 22:24:05.614391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:05.614391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:24:05.614391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:05.614391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:05.614391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:05.614391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:05.614391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:05.616564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:05.616564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:05.616564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:05.616564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:05.616564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:05.616564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:05.616564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:05.616564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:24:05.616564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:24:05.616564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:24:05.616564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:24:05.616564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:05.616564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:05.616564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:05.616564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:05.616564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:05.617423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:05.617423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:05.617423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:05.617423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:05.617423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:05.617423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:05.617423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:05.617423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:05.617423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:05.617423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:05.617423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:05.617423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:24:05.617423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:05.617423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 22:24:05.617423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:05.619001 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 462/900] 10.222.0.43 () {56 vars in 1132 bytes} [Thu Jun 23 22:24:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:05.620072 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf97239-5b1d-489d-b8fb-4cd1528abe72 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:05.625566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 439/901] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:05.627198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887439c-2577-49cf-b88f-3d834f3d3241 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:05.687341 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:05.730538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf97239-5b1d-489d-b8fb-4cd1528abe72 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:05.746898 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887439c-2577-49cf-b88f-3d834f3d3241 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:05.761700 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:05.766957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a1f94-0c8e-4254-8f87-6ec5e4601862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:05.767623 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a1f94-0c8e-4254-8f87-6ec5e4601862 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:05.767906 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a1f94-0c8e-4254-8f87-6ec5e4601862 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:05.768181 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a1f94-0c8e-4254-8f87-6ec5e4601862 None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:05.769000 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a1f94-0c8e-4254-8f87-6ec5e4601862 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:05.769310 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a1f94-0c8e-4254-8f87-6ec5e4601862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:05.770671 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a1f94-0c8e-4254-8f87-6ec5e4601862 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:05.776601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 463/902] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 22:24:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:05.780545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609fc3a-62ee-4811-a1ae-7957c58b5e03 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:05.781404 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609fc3a-62ee-4811-a1ae-7957c58b5e03 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:05.782087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609fc3a-62ee-4811-a1ae-7957c58b5e03 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:05.783584 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 464/903] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:05.784446 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609fc3a-62ee-4811-a1ae-7957c58b5e03 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:05.789939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb11cf-620a-4369-a574-6d44707b7dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:05.790621 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb11cf-620a-4369-a574-6d44707b7dbd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:05.790912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb11cf-620a-4369-a574-6d44707b7dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:05.791240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb11cf-620a-4369-a574-6d44707b7dbd None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:05.791977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb11cf-620a-4369-a574-6d44707b7dbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:05.792289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb11cf-620a-4369-a574-6d44707b7dbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:05.793665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb11cf-620a-4369-a574-6d44707b7dbd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:05.806352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 440/904] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:05.808019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b470a7-7863-4130-935f-39e74360fe07 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:05.808601 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b470a7-7863-4130-935f-39e74360fe07 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:05.808601 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b470a7-7863-4130-935f-39e74360fe07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:05.825929 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c9b470a7-7863-4130-935f-39e74360fe07 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:05.827059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c9b470a7-7863-4130-935f-39e74360fe07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:05.903125 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609fc3a-62ee-4811-a1ae-7957c58b5e03 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:05.906254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 441/905] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:05.907188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b470a7-7863-4130-935f-39e74360fe07 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:05.941279 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:05.952099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2647fd2-79e0-474c-aab8-0f0b885465d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:05.952775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2647fd2-79e0-474c-aab8-0f0b885465d4 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:05.953065 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2647fd2-79e0-474c-aab8-0f0b885465d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:05.953354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2647fd2-79e0-474c-aab8-0f0b885465d4 None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:05.954210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2647fd2-79e0-474c-aab8-0f0b885465d4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:05.954379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2647fd2-79e0-474c-aab8-0f0b885465d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:05.955758 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2647fd2-79e0-474c-aab8-0f0b885465d4 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:05.961264 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 465/906] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 22:24:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:05.962019 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2647fd2-79e0-474c-aab8-0f0b885465d4 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:06.028045 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b470a7-7863-4130-935f-39e74360fe07 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:06.029564 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:06.065488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c17c2-c5e2-4b8a-a9c6-18979c0832af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:06.066289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c17c2-c5e2-4b8a-a9c6-18979c0832af None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:06.066575 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c17c2-c5e2-4b8a-a9c6-18979c0832af None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:06.066863 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c17c2-c5e2-4b8a-a9c6-18979c0832af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:06.069388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2647fd2-79e0-474c-aab8-0f0b885465d4 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:06.074592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c17c2-c5e2-4b8a-a9c6-18979c0832af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:06.076141 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c17c2-c5e2-4b8a-a9c6-18979c0832af None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:06.091051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-5d1c17c2-c5e2-4b8a-a9c6-18979c0832af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:24:06.091051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:06.091051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:06.091051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:06.091051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:06.091051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:06.091051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:06.091051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:06.091051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:06.091051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:06.091051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:06.091051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:06.091051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:24:06.091051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:24:06.091051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:24:06.091975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:24:06.091975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:06.091975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:06.091975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:06.091975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:06.091975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:06.091975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:06.091975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:06.091975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:06.091975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:06.091975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:06.091975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:06.091975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:06.091975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:06.091975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:06.091975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:06.091975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:06.092872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:24:06.092872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:24:06.092872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:24:06.092872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:24:06.092872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:24:06.092872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:24:06.092872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:24:06.092872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:06.092872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 442/907] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:24:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:06.093372 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c17c2-c5e2-4b8a-a9c6-18979c0832af None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:06.110187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b05c57-28b7-4294-b35e-d4d23be9e975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:06.110871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b05c57-28b7-4294-b35e-d4d23be9e975 None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:06.111187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b05c57-28b7-4294-b35e-d4d23be9e975 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:06.111475 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b05c57-28b7-4294-b35e-d4d23be9e975 None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:06.112203 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b05c57-28b7-4294-b35e-d4d23be9e975 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:06.113571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b05c57-28b7-4294-b35e-d4d23be9e975 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:06.139093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 466/908] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 22:24:05 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:24:06.140009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b05c57-28b7-4294-b35e-d4d23be9e975 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:06.209472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:06.246427 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c17c2-c5e2-4b8a-a9c6-18979c0832af None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:06.293462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b05c57-28b7-4294-b35e-d4d23be9e975 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:06.303126 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ec1d8-3f33-411d-bda0-e7bbeebec2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:06.304355 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ec1d8-3f33-411d-bda0-e7bbeebec2fd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:06.304644 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ec1d8-3f33-411d-bda0-e7bbeebec2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:06.305531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ec1d8-3f33-411d-bda0-e7bbeebec2fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:06.305724 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ec1d8-3f33-411d-bda0-e7bbeebec2fd None admin] RBAC: Adding query filter params (interface=public, service_id=e24fe3cdc86749b8b0f82c2475130f6f, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:06.306444 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ec1d8-3f33-411d-bda0-e7bbeebec2fd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:06.308209 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ec1d8-3f33-411d-bda0-e7bbeebec2fd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:06.316642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 443/909] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Jun 23 22:24:06 2022] GET /identity/v3/endpoints?service_id=e24fe3cdc86749b8b0f82c2475130f6f&interface=public®ion_id=RegionOne => generated 194 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:06.322020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ec1d8-3f33-411d-bda0-e7bbeebec2fd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:06.338541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf92dd-46ff-4244-92c5-f24ec6f8ef18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:06.339702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf92dd-46ff-4244-92c5-f24ec6f8ef18 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:06.339702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf92dd-46ff-4244-92c5-f24ec6f8ef18 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:06.339702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf92dd-46ff-4244-92c5-f24ec6f8ef18 None admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:06.340508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf92dd-46ff-4244-92c5-f24ec6f8ef18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:06.340821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf92dd-46ff-4244-92c5-f24ec6f8ef18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:06.342241 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf92dd-46ff-4244-92c5-f24ec6f8ef18 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:06.357629 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 467/910] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:24:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:06.359631 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:06.367159 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf92dd-46ff-4244-92c5-f24ec6f8ef18 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:06.443583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ec1d8-3f33-411d-bda0-e7bbeebec2fd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:06.478205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf92dd-46ff-4244-92c5-f24ec6f8ef18 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:06.479932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dff47-16cc-457c-982e-a990a60f20a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:06.480712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dff47-16cc-457c-982e-a990a60f20a4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:06.481699 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dff47-16cc-457c-982e-a990a60f20a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:06.481699 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dff47-16cc-457c-982e-a990a60f20a4 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:06.482348 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42dff47-16cc-457c-982e-a990a60f20a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:06.483854 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42dff47-16cc-457c-982e-a990a60f20a4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:06.487050 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:06.493400 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 444/911] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Jun 23 22:24:06 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:24:06.519102 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2de418-6fde-4d4b-8c35-5a420e23be14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:06.520052 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2de418-6fde-4d4b-8c35-5a420e23be14 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:06.520488 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2de418-6fde-4d4b-8c35-5a420e23be14 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:06.520944 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2de418-6fde-4d4b-8c35-5a420e23be14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:06.522685 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2de418-6fde-4d4b-8c35-5a420e23be14 None admin] RBAC: Adding query filter params (role.id=26dbcfd7d83742b1b1cd311915a9c0a8, scope.project.id=56da0975019c44409b53be473131143a, user.id=df1aeb2be41c49819b811ecfa47d4155) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:06.523481 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2de418-6fde-4d4b-8c35-5a420e23be14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:06.525120 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2de418-6fde-4d4b-8c35-5a420e23be14 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:06.532215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 468/912] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:24:06 2022] GET /identity/v3/role_assignments?user.id=df1aeb2be41c49819b811ecfa47d4155&scope.project.id=56da0975019c44409b53be473131143a&role.id=26dbcfd7d83742b1b1cd311915a9c0a8 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:07.171777 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe9508-59cf-4125-ae67-eda1a1fd789b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:07.172288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe9508-59cf-4125-ae67-eda1a1fd789b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:07.172365 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe9508-59cf-4125-ae67-eda1a1fd789b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:07.173972 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 445/913] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:07.180446 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdc394-37f1-4cae-96af-a2df7b170dfe None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:07.181008 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdc394-37f1-4cae-96af-a2df7b170dfe None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:07.181449 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bdc394-37f1-4cae-96af-a2df7b170dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:07.200455 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-31bdc394-37f1-4cae-96af-a2df7b170dfe None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:07.201655 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-31bdc394-37f1-4cae-96af-a2df7b170dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:07.270389 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 469/914] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:07.294282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe9508-59cf-4125-ae67-eda1a1fd789b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:07.390955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:07.402367 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe9508-59cf-4125-ae67-eda1a1fd789b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:07.438149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74542938-6471-4b52-a280-6cc3bd9d1743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:07.438969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74542938-6471-4b52-a280-6cc3bd9d1743 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:07.439263 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74542938-6471-4b52-a280-6cc3bd9d1743 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:07.439551 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74542938-6471-4b52-a280-6cc3bd9d1743 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:07.440208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74542938-6471-4b52-a280-6cc3bd9d1743 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:07.441591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74542938-6471-4b52-a280-6cc3bd9d1743 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:07.454218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-74542938-6471-4b52-a280-6cc3bd9d1743 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 22:24:07.454218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:07.454218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:24:07.454218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:24:07.454218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:24:07.454218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:24:07.454218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:24:07.454218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:07.454218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 22:24:07.454218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:07.454218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:24:07.454218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:07.454218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:07.454218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:07.454218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:07.454218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:07.454218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:07.455240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:07.455240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:07.455240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:07.455240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:07.455240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:07.455240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:07.455240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:24:07.455240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:24:07.455240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:24:07.455240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:24:07.455240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:07.455240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:07.455240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:07.455240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:07.455240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:07.455240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:07.456110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:07.456110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:07.456110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:07.456110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:07.456110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:07.456110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:07.456110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:07.456110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:07.456110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:07.456110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:07.456110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:24:07.456110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:07.456110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 22:24:07.456110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:07.456110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 446/915] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:24:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:07.462333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdc394-37f1-4cae-96af-a2df7b170dfe None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:07.561690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:07.564766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bdc394-37f1-4cae-96af-a2df7b170dfe None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:07.601563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699d61d-c4bb-4468-ba7c-e9f1f50ef1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:07.602242 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699d61d-c4bb-4468-ba7c-e9f1f50ef1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:07.602531 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699d61d-c4bb-4468-ba7c-e9f1f50ef1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:07.602815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699d61d-c4bb-4468-ba7c-e9f1f50ef1e1 None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:07.603553 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699d61d-c4bb-4468-ba7c-e9f1f50ef1e1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:07.603862 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699d61d-c4bb-4468-ba7c-e9f1f50ef1e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:07.605233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699d61d-c4bb-4468-ba7c-e9f1f50ef1e1 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:07.610413 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 470/916] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:24:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:07.616259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74542938-6471-4b52-a280-6cc3bd9d1743 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:07.726756 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74542938-6471-4b52-a280-6cc3bd9d1743 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:07.761813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101c5bc-9ad1-42b0-90d7-1b15e3d7bca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:07.762476 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101c5bc-9ad1-42b0-90d7-1b15e3d7bca4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:07.762762 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101c5bc-9ad1-42b0-90d7-1b15e3d7bca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:07.763073 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101c5bc-9ad1-42b0-90d7-1b15e3d7bca4 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:07.763776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101c5bc-9ad1-42b0-90d7-1b15e3d7bca4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:07.764161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101c5bc-9ad1-42b0-90d7-1b15e3d7bca4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:07.765537 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101c5bc-9ad1-42b0-90d7-1b15e3d7bca4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:07.770801 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 447/917] 10.222.0.43 () {56 vars in 1114 bytes} [Thu Jun 23 22:24:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:07.776400 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699d61d-c4bb-4468-ba7c-e9f1f50ef1e1 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:07.884738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699d61d-c4bb-4468-ba7c-e9f1f50ef1e1 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:07.920709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c217ad8-6bdc-476c-bda8-80ec299f080b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:07.921376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c217ad8-6bdc-476c-bda8-80ec299f080b None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:07.921666 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c217ad8-6bdc-476c-bda8-80ec299f080b None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:07.921947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c217ad8-6bdc-476c-bda8-80ec299f080b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:07.922696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c217ad8-6bdc-476c-bda8-80ec299f080b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:07.924099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c217ad8-6bdc-476c-bda8-80ec299f080b None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:07.946028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 471/918] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Jun 23 22:24:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:24:08.404363 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0aaf64-077c-4b20-8216-5ac57c16e5fc None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:08.405386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0aaf64-077c-4b20-8216-5ac57c16e5fc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:08.405883 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0aaf64-077c-4b20-8216-5ac57c16e5fc None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:08.407753 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 448/919] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:08.413396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c4de3-2276-44c1-9aa8-194b694221bb None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:08.414224 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c4de3-2276-44c1-9aa8-194b694221bb None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:08.414722 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c4de3-2276-44c1-9aa8-194b694221bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:08.432768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-a04c4de3-2276-44c1-9aa8-194b694221bb None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:08.434306 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-a04c4de3-2276-44c1-9aa8-194b694221bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:08.449510 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:08.513792 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 472/920] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:08.537166 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0aaf64-077c-4b20-8216-5ac57c16e5fc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:08.562197 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:08.649553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0aaf64-077c-4b20-8216-5ac57c16e5fc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:08.673382 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd638133-d88a-4b9a-b017-fd11fadcff35 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:08.673668 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd638133-d88a-4b9a-b017-fd11fadcff35 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:08.673974 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd638133-d88a-4b9a-b017-fd11fadcff35 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:08.675404 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 473/921] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:08.676532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433aa95-ec52-42a4-906a-82f27ff251d7 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:08.676882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433aa95-ec52-42a4-906a-82f27ff251d7 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:08.677178 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433aa95-ec52-42a4-906a-82f27ff251d7 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:08.679302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 474/922] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:08.682816 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a448f-60b3-4916-a821-a2de0f2c6a6a None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:08.683197 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a448f-60b3-4916-a821-a2de0f2c6a6a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:08.683485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a448f-60b3-4916-a821-a2de0f2c6a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:08.685813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c5fda-5063-415a-8f97-6c2e053289ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:08.686627 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c5fda-5063-415a-8f97-6c2e053289ce None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:08.686936 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c5fda-5063-415a-8f97-6c2e053289ce None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:08.687231 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c5fda-5063-415a-8f97-6c2e053289ce None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:08.687894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c5fda-5063-415a-8f97-6c2e053289ce None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:08.689272 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c5fda-5063-415a-8f97-6c2e053289ce None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:08.700749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-d93a448f-60b3-4916-a821-a2de0f2c6a6a None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:08.701969 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d93a448f-60b3-4916-a821-a2de0f2c6a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:08.702869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-272c5fda-5063-415a-8f97-6c2e053289ce None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:24:08.702869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:08.702869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:24:08.702869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:24:08.702869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:24:08.702869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:24:08.702869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:24:08.702869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:08.702869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:24:08.702869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:08.702869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:24:08.702869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:08.702869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:08.702869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:08.702869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:08.702869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:08.702869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:08.703857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:08.703857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:08.703857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:08.703857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:08.703857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:08.703857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:08.703857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:24:08.703857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:24:08.703857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:24:08.703857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:24:08.703857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:08.703857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:08.703857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:08.703857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:08.703857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:08.703857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:08.704702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:08.704702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:08.704702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:08.704702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:08.704702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:08.704702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:08.704702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:08.704702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:08.704702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:08.704702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:08.704702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:24:08.704702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:08.704702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:24:08.704702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:08.704702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 449/923] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 22:24:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:08.705510 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b6477-1afb-4933-8a13-52853dc604bf None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:08.705611 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b6477-1afb-4933-8a13-52853dc604bf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:08.705897 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b6477-1afb-4933-8a13-52853dc604bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:08.729295 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4e5b6477-1afb-4933-8a13-52853dc604bf None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:08.730468 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4e5b6477-1afb-4933-8a13-52853dc604bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:08.780612 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 475/924] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:08.781839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a448f-60b3-4916-a821-a2de0f2c6a6a None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:08.807691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 450/925] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:08.808855 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b6477-1afb-4933-8a13-52853dc604bf None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:08.831152 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:08.862364 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:08.887376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a448f-60b3-4916-a821-a2de0f2c6a6a None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:08.923629 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60845a-9010-4b7c-aca4-df27ba4cc19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:08.923801 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b6477-1afb-4933-8a13-52853dc604bf None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:08.924248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60845a-9010-4b7c-aca4-df27ba4cc19b None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:08.924536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60845a-9010-4b7c-aca4-df27ba4cc19b None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:08.924813 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60845a-9010-4b7c-aca4-df27ba4cc19b None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:08.925510 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60845a-9010-4b7c-aca4-df27ba4cc19b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:08.925816 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60845a-9010-4b7c-aca4-df27ba4cc19b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:08.927214 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60845a-9010-4b7c-aca4-df27ba4cc19b None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:08.936578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 476/926] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 22:24:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:08.936578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60845a-9010-4b7c-aca4-df27ba4cc19b None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:08.964698 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3681cf-54bc-4ebb-9e01-1cdd31622ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:08.965544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3681cf-54bc-4ebb-9e01-1cdd31622ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:08.965791 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3681cf-54bc-4ebb-9e01-1cdd31622ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:08.966071 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3681cf-54bc-4ebb-9e01-1cdd31622ae3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:08.966726 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3681cf-54bc-4ebb-9e01-1cdd31622ae3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:08.968119 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3681cf-54bc-4ebb-9e01-1cdd31622ae3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:08.982302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ae3681cf-54bc-4ebb-9e01-1cdd31622ae3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 22:24:08.982302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:08.982302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:24:08.982302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:24:08.982302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:24:08.982302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:24:08.982302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:24:08.982302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:08.982302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 22:24:08.982302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:08.982302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:24:08.982302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:08.982302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:08.982302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:08.982302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:08.982302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:08.983507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:08.983507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:08.983507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:08.983507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:08.983507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:08.983507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:08.983507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:08.983507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:24:08.983507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:24:08.983507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:24:08.983507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:24:08.983507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:08.983507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:08.983507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:08.983507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:08.983507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:08.984366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:08.984366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:08.984366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:08.984366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:08.984366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:08.984366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:08.984366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:08.984366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:08.984366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:08.984366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:08.984366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:08.984366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:24:08.984366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:08.984366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 22:24:08.984366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:08.984366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 451/927] 10.222.0.43 () {56 vars in 1132 bytes} [Thu Jun 23 22:24:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:08.985222 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3681cf-54bc-4ebb-9e01-1cdd31622ae3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:09.048088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60845a-9010-4b7c-aca4-df27ba4cc19b None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:09.067661 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:09.085355 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7772fe08-61fa-4879-a637-c3ccd9a2a3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:09.086171 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772fe08-61fa-4879-a637-c3ccd9a2a3cc None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:09.086460 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772fe08-61fa-4879-a637-c3ccd9a2a3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:09.086744 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772fe08-61fa-4879-a637-c3ccd9a2a3cc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:09.094734 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:09.094854 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3681cf-54bc-4ebb-9e01-1cdd31622ae3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:09.098432 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7772fe08-61fa-4879-a637-c3ccd9a2a3cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:24:09.108562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-7772fe08-61fa-4879-a637-c3ccd9a2a3cc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:24:09.108562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:09.108562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:09.108562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:09.108562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:09.108562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:09.108562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:09.108562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:09.108562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:09.108562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:09.108562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:09.108562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:09.108562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:24:09.108562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:24:09.108562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:24:09.108562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:24:09.109532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:24:09.109532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:24:09.109532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:24:09.109532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:24:09.109532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:09.109532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:09.109532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:09.109532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:09.109532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:09.109532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:09.109532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:09.109532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:09.109532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:09.109532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:09.109532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:09.109532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:09.110369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:09.110369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:09.110369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:09.110369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:09.110369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:24:09.110369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:24:09.110369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:24:09.110369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:24:09.110369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:24:09.110369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:24:09.110369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:24:09.110369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:09.113427 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 477/928] 10.222.0.43 () {56 vars in 1120 bytes} [Thu Jun 23 22:24:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:09.119414 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7772fe08-61fa-4879-a637-c3ccd9a2a3cc None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:09.137122 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5eb3d-9bcd-413e-96b4-c66c146a829a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:09.137796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5eb3d-9bcd-413e-96b4-c66c146a829a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:09.138087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5eb3d-9bcd-413e-96b4-c66c146a829a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:09.138374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5eb3d-9bcd-413e-96b4-c66c146a829a None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:09.139158 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5eb3d-9bcd-413e-96b4-c66c146a829a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:09.139435 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5eb3d-9bcd-413e-96b4-c66c146a829a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:09.140855 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5eb3d-9bcd-413e-96b4-c66c146a829a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:09.146572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 452/929] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 22:24:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:09.147267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5eb3d-9bcd-413e-96b4-c66c146a829a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:09.242498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7772fe08-61fa-4879-a637-c3ccd9a2a3cc None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:09.255725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5eb3d-9bcd-413e-96b4-c66c146a829a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:09.284044 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c0252-9193-49a3-98b2-8a6e51ca93c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:09.284710 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c0252-9193-49a3-98b2-8a6e51ca93c9 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:09.284994 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c0252-9193-49a3-98b2-8a6e51ca93c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:09.285274 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c0252-9193-49a3-98b2-8a6e51ca93c9 None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:09.285966 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c0252-9193-49a3-98b2-8a6e51ca93c9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:09.286422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c0252-9193-49a3-98b2-8a6e51ca93c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:09.288231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c0252-9193-49a3-98b2-8a6e51ca93c9 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:09.292256 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48af1a-0999-463c-a210-3e926f99eef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:09.297540 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48af1a-0999-463c-a210-3e926f99eef7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:09.297667 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 478/930] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 22:24:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:09.297767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48af1a-0999-463c-a210-3e926f99eef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:09.297767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48af1a-0999-463c-a210-3e926f99eef7 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:09.297899 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c0252-9193-49a3-98b2-8a6e51ca93c9 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:09.297968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48af1a-0999-463c-a210-3e926f99eef7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:09.297968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48af1a-0999-463c-a210-3e926f99eef7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:09.297968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48af1a-0999-463c-a210-3e926f99eef7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:09.310266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 453/931] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Jun 23 22:24:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:09.310266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48af1a-0999-463c-a210-3e926f99eef7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:09.310266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:09.368483 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:09.410656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c0252-9193-49a3-98b2-8a6e51ca93c9 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:09.419621 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48af1a-0999-463c-a210-3e926f99eef7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:09.447682 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec1086-91fd-4eff-8df5-71141b8384c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:09.448355 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec1086-91fd-4eff-8df5-71141b8384c9 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:09.448645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec1086-91fd-4eff-8df5-71141b8384c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:09.448933 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec1086-91fd-4eff-8df5-71141b8384c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:09.449725 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec1086-91fd-4eff-8df5-71141b8384c9 None admin] RBAC: Adding query filter params (interface=public, service_id=b04f1788e9734fe4a3e334440f7889a0, region_id=RegionOne) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:09.450047 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec1086-91fd-4eff-8df5-71141b8384c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:09.451488 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec1086-91fd-4eff-8df5-71141b8384c9 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:09.456717 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3341ec-7404-463f-9857-e0f6cb67700e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:09.456857 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 479/932] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Jun 23 22:24:09 2022] GET /identity/v3/endpoints?service_id=b04f1788e9734fe4a3e334440f7889a0&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:09.457760 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3341ec-7404-463f-9857-e0f6cb67700e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:09.458115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3341ec-7404-463f-9857-e0f6cb67700e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:09.458466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3341ec-7404-463f-9857-e0f6cb67700e None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:09.459415 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3341ec-7404-463f-9857-e0f6cb67700e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:09.459805 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3341ec-7404-463f-9857-e0f6cb67700e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:09.461199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3341ec-7404-463f-9857-e0f6cb67700e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:09.462457 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec1086-91fd-4eff-8df5-71141b8384c9 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:09.466501 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 454/933] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 22:24:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:09.467099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3341ec-7404-463f-9857-e0f6cb67700e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:09.537181 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:09.573741 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec1086-91fd-4eff-8df5-71141b8384c9 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:09.577367 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3341ec-7404-463f-9857-e0f6cb67700e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:09.600287 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:09.609949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091a8b1-91e6-43d3-8f4c-21c1b3c8ffe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:09.610753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091a8b1-91e6-43d3-8f4c-21c1b3c8ffe9 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:09.611079 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091a8b1-91e6-43d3-8f4c-21c1b3c8ffe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:09.611365 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091a8b1-91e6-43d3-8f4c-21c1b3c8ffe9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:09.614420 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24c9c6-1d72-4bd4-8535-a2c1273c0f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:09.615315 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24c9c6-1d72-4bd4-8535-a2c1273c0f42 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:09.615683 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24c9c6-1d72-4bd4-8535-a2c1273c0f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:09.615986 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24c9c6-1d72-4bd4-8535-a2c1273c0f42 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:09.616684 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24c9c6-1d72-4bd4-8535-a2c1273c0f42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:09.618064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24c9c6-1d72-4bd4-8535-a2c1273c0f42 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:09.623087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091a8b1-91e6-43d3-8f4c-21c1b3c8ffe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:09.635176 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 455/934] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Jun 23 22:24:09 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:24:09.635409 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091a8b1-91e6-43d3-8f4c-21c1b3c8ffe9 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:09.635514 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24c9c6-1d72-4bd4-8535-a2c1273c0f42 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:09.658196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-e091a8b1-91e6-43d3-8f4c-21c1b3c8ffe9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:09.658196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:09.658196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:09.658196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:09.658196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:09.658196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:09.658196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:09.658196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:09.658196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:09.658196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:09.658196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:09.658196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:09.658196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:24:09.658196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:24:09.658196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:24:09.659196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:24:09.659196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:09.659196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:09.659196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:09.659196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:09.659196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:09.659196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:09.659196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:09.659196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:09.659196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:09.659196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:09.659196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:09.659196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:09.659196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:09.659196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:09.659196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:09.659196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:09.660169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:24:09.660169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:24:09.660169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:09.660169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:09.661628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 480/935] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:09.669114 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091a8b1-91e6-43d3-8f4c-21c1b3c8ffe9 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:09.754599 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24c9c6-1d72-4bd4-8535-a2c1273c0f42 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:09.790895 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:09.798250 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f319fe-4a1f-4c7d-8226-b27df4d6d557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:09.799147 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f319fe-4a1f-4c7d-8226-b27df4d6d557 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:09.799572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f319fe-4a1f-4c7d-8226-b27df4d6d557 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:09.799926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f319fe-4a1f-4c7d-8226-b27df4d6d557 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:09.800748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f319fe-4a1f-4c7d-8226-b27df4d6d557 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:09.804023 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f319fe-4a1f-4c7d-8226-b27df4d6d557 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:09.811668 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091a8b1-91e6-43d3-8f4c-21c1b3c8ffe9 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:09.833437 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 456/936] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Jun 23 22:24:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:24:09.869711 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe5a20-a50d-46ad-b9ae-c787f1474d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:09.870461 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe5a20-a50d-46ad-b9ae-c787f1474d5c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:09.870754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe5a20-a50d-46ad-b9ae-c787f1474d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:09.871136 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe5a20-a50d-46ad-b9ae-c787f1474d5c None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:09.871887 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe5a20-a50d-46ad-b9ae-c787f1474d5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:09.872197 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe5a20-a50d-46ad-b9ae-c787f1474d5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:09.873608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe5a20-a50d-46ad-b9ae-c787f1474d5c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:09.888973 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 481/937] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:09.896615 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f319fe-4a1f-4c7d-8226-b27df4d6d557 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:10.014042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f319fe-4a1f-4c7d-8226-b27df4d6d557 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:10.052785 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6c9af-3094-40d4-9913-f562ec45de68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:10.053618 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6c9af-3094-40d4-9913-f562ec45de68 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:10.053910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6c9af-3094-40d4-9913-f562ec45de68 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:10.054191 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6c9af-3094-40d4-9913-f562ec45de68 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:10.062562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6c9af-3094-40d4-9913-f562ec45de68 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:10.064048 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6c9af-3094-40d4-9913-f562ec45de68 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:10.079968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-88c6c9af-3094-40d4-9913-f562ec45de68 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:24:10.079968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:10.079968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:10.079968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:10.079968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:10.079968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:10.079968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:10.079968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:10.079968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:10.079968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:10.079968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:10.079968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:10.079968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:24:10.079968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:24:10.079968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:24:10.079968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:24:10.082116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:10.082116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:10.082116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:24:10.082116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:24:10.082116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:24:10.082116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:24:10.082116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:10.082116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:10.082116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:10.082116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:10.082116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:10.082116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:10.082116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:10.082116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:10.082116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:10.082116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:10.082116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:10.082116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:10.084745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:10.084745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:10.084745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:24:10.084745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:24:10.084745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:24:10.084745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:24:10.084745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:24:10.084745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:24:10.084745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:24:10.084745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:10.084745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 457/938] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 22:24:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:10.090951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe5a20-a50d-46ad-b9ae-c787f1474d5c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:10.104623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f654ad-ec58-4246-8d99-c0377f7a7f20 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:10.105310 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f654ad-ec58-4246-8d99-c0377f7a7f20 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:10.105794 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f654ad-ec58-4246-8d99-c0377f7a7f20 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:10.107650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 458/939] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:24:10.112879 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942a00c-f7d0-4e3d-9a8b-ea19bcb0cd33 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:10.113430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942a00c-f7d0-4e3d-9a8b-ea19bcb0cd33 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:10.113929 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942a00c-f7d0-4e3d-9a8b-ea19bcb0cd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:10.133669 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1942a00c-f7d0-4e3d-9a8b-ea19bcb0cd33 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:10.135155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1942a00c-f7d0-4e3d-9a8b-ea19bcb0cd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:10.137781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:10.184669 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:10.216161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe5a20-a50d-46ad-b9ae-c787f1474d5c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:10.216873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 459/940] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:10.240054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942a00c-f7d0-4e3d-9a8b-ea19bcb0cd33 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:10.252878 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a24cc-cf44-4873-a4ab-f91bee14ac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:10.253556 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a24cc-cf44-4873-a4ab-f91bee14ac4a None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:10.253841 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a24cc-cf44-4873-a4ab-f91bee14ac4a None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:10.254122 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a24cc-cf44-4873-a4ab-f91bee14ac4a None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:10.254987 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a24cc-cf44-4873-a4ab-f91bee14ac4a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:10.255304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a24cc-cf44-4873-a4ab-f91bee14ac4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:10.257875 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a24cc-cf44-4873-a4ab-f91bee14ac4a None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:10.279938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 482/941] 10.222.0.43 () {56 vars in 1149 bytes} [Thu Jun 23 22:24:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:10.285132 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a24cc-cf44-4873-a4ab-f91bee14ac4a None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:10.356645 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942a00c-f7d0-4e3d-9a8b-ea19bcb0cd33 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:10.393454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811224c3-519f-4ccb-929b-155fa0aff624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:10.394118 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811224c3-519f-4ccb-929b-155fa0aff624 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:10.394408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811224c3-519f-4ccb-929b-155fa0aff624 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:10.394757 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811224c3-519f-4ccb-929b-155fa0aff624 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:10.395732 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811224c3-519f-4ccb-929b-155fa0aff624 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:10.397241 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811224c3-519f-4ccb-929b-155fa0aff624 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:10.399175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: WARNING keystone.server.flask.common [None req-811224c3-519f-4ccb-929b-155fa0aff624 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 23 22:24:10.415069 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:10.422007 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.sql.core [None req-811224c3-519f-4ccb-929b-155fa0aff624 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 22:24:10.422007 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 22:24:10.422007 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [parameters: {'id': '3413f641a2d34a58abd5425a3a42fb78', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 22:24:10.422007 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80203) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 22:24:10.422940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a24cc-cf44-4873-a4ab-f91bee14ac4a None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:10.431112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-811224c3-519f-4ccb-929b-155fa0aff624 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3413f641a2d34a58abd5425a3a42fb78.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3413f641a2d34a58abd5425a3a42fb78. Jun 23 22:24:10.431112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:10.431112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 22:24:10.431112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 22:24:10.431112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 22:24:10.431112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 22:24:10.431112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 22:24:10.431112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 22:24:10.431112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 22:24:10.431112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) Jun 23 22:24:10.431112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 22:24:10.431112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 22:24:10.431112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 22:24:10.432078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() Jun 23 22:24:10.432078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 22:24:10.432078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 22:24:10.432078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 22:24:10.432078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 22:24:10.432078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 22:24:10.432078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 22:24:10.432078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 22:24:10.432078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 22:24:10.432078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 22:24:10.432078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:10.432078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 22:24:10.432078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:10.432078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:10.432078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 22:24:10.432078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 22:24:10.432078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 23 22:24:10.432078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 23 22:24:10.433211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 22:24:10.433211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) Jun 23 22:24:10.433211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 22:24:10.433211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application yield resource Jun 23 22:24:10.433211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 22:24:10.433211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) Jun 23 22:24:10.433211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 22:24:10.433211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 22:24:10.433211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 22:24:10.433211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 22:24:10.433211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 22:24:10.433211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise self.value Jun 23 22:24:10.433211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 22:24:10.433211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 22:24:10.433211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 22:24:10.433211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application session.commit() Jun 23 22:24:10.433211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 22:24:10.434266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 22:24:10.434266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 22:24:10.434266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 22:24:10.434266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 22:24:10.434266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.flush() Jun 23 22:24:10.434266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 22:24:10.434266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 22:24:10.434266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 22:24:10.434266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 22:24:10.434266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 22:24:10.434266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application compat.raise_( Jun 23 22:24:10.434266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 22:24:10.434266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception Jun 23 22:24:10.434266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 22:24:10.434266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 22:24:10.434266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 22:24:10.434266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 22:24:10.435865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 22:24:10.435865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 22:24:10.435865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 22:24:10.435865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 22:24:10.435865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 22:24:10.435865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 22:24:10.435865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 22:24:10.435865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 22:24:10.435865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 22:24:10.435865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 22:24:10.435865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 22:24:10.435865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 22:24:10.435865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 22:24:10.435865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 22:24:10.435865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 22:24:10.437670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 22:24:10.437670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 22:24:10.437670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception Jun 23 22:24:10.437670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 22:24:10.437670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 22:24:10.437670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 22:24:10.437670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 22:24:10.437670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 22:24:10.437670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 22:24:10.437670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 22:24:10.437670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) Jun 23 22:24:10.437670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 22:24:10.437670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 22:24:10.437670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 22:24:10.437670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() Jun 23 22:24:10.437670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 22:24:10.437670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 22:24:10.447328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 22:24:10.447328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 22:24:10.447328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 22:24:10.447328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 22:24:10.447328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 22:24:10.447328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 22:24:10.447328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 22:24:10.447328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 22:24:10.447328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [parameters: {'id': '3413f641a2d34a58abd5425a3a42fb78', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 22:24:10.447328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 22:24:10.447328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:10.447328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:24:10.447328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:10.447328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:10.447328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 23 22:24:10.449020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 23 22:24:10.449020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 22:24:10.449020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 22:24:10.449020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 23 22:24:10.449020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:10.449020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:24:10.449020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:10.449020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:10.449020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:10.449020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:10.449020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:10.449020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:10.449020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:10.449020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:10.449020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:10.449020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:10.449020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:10.451252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:10.451252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 23 22:24:10.451252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 23 22:24:10.451252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:10.451252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:10.451252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 23 22:24:10.451252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 23 22:24:10.451252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3413f641a2d34a58abd5425a3a42fb78. Jun 23 22:24:10.451252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:10.451252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 460/942] 10.222.0.43 () {60 vars in 1140 bytes} [Thu Jun 23 22:24:10 2022] POST /identity/v3/projects => generated 262 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:10.451252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811224c3-519f-4ccb-929b-155fa0aff624 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:10.467467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46528a34-e6db-46a4-98a6-6a5c965a7b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:10.468334 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528a34-e6db-46a4-98a6-6a5c965a7b55 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:10.468625 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528a34-e6db-46a4-98a6-6a5c965a7b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:10.468907 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528a34-e6db-46a4-98a6-6a5c965a7b55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:10.473759 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:10.481729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46528a34-e6db-46a4-98a6-6a5c965a7b55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:10.483239 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46528a34-e6db-46a4-98a6-6a5c965a7b55 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:10.504572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-46528a34-e6db-46a4-98a6-6a5c965a7b55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:10.504572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:10.504572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:10.504572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:10.504572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:10.504572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:10.504572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:10.504572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:10.504572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:10.504572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:10.504572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:10.504572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:10.504572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:24:10.504572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:24:10.504572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:24:10.507222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:24:10.507222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:10.507222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:10.507222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:10.507222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:10.507222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:10.507222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:10.507222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:10.507222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:10.507222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:10.507222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:10.507222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:10.507222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:10.507222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:10.507222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:10.507222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:10.507222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:10.509346 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:24:10.509346 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:24:10.509346 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:10.509346 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:10.513261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 483/943] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:10.513261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46528a34-e6db-46a4-98a6-6a5c965a7b55 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:10.551917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811224c3-519f-4ccb-929b-155fa0aff624 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:10.590025 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01f3d4-9a2f-400d-8172-c56c9bf302af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:10.590828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01f3d4-9a2f-400d-8172-c56c9bf302af None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:10.591243 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01f3d4-9a2f-400d-8172-c56c9bf302af None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:10.591529 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01f3d4-9a2f-400d-8172-c56c9bf302af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:10.598803 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01f3d4-9a2f-400d-8172-c56c9bf302af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:10.600651 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01f3d4-9a2f-400d-8172-c56c9bf302af None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:10.616079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-eb01f3d4-9a2f-400d-8172-c56c9bf302af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:24:10.616079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:10.616079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:10.616079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:10.616079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:10.616079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:10.616079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:10.616079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:10.616079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:10.616079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:10.616079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:10.616079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:10.616079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:24:10.616079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:24:10.616079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:24:10.618111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:24:10.618111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:10.618111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:10.618111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:10.618111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:10.618111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:10.618111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:10.618111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:10.618111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:10.618111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:10.618111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:10.618111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:10.618111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:10.618111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:10.618111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:10.618111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:10.618111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:10.620286 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:24:10.620286 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:24:10.620286 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:24:10.620286 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:24:10.620286 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:24:10.620286 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:24:10.620286 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:24:10.620286 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:10.626945 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 461/944] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:24:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:10.626945 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01f3d4-9a2f-400d-8172-c56c9bf302af None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:10.627311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46528a34-e6db-46a4-98a6-6a5c965a7b55 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:10.662468 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb382ee-29a3-4af6-bda9-55db5201d865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:10.663089 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb382ee-29a3-4af6-bda9-55db5201d865 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:10.663381 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb382ee-29a3-4af6-bda9-55db5201d865 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:10.663664 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb382ee-29a3-4af6-bda9-55db5201d865 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:10.664439 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb382ee-29a3-4af6-bda9-55db5201d865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:10.664777 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb382ee-29a3-4af6-bda9-55db5201d865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:10.666239 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb382ee-29a3-4af6-bda9-55db5201d865 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:10.684304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 484/945] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:10.685695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb382ee-29a3-4af6-bda9-55db5201d865 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:10.696145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:10.722712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:10.733485 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01f3d4-9a2f-400d-8172-c56c9bf302af None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:10.771013 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7cefa6-48f0-4cbf-9e0f-9e22c9090776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:10.771878 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7cefa6-48f0-4cbf-9e0f-9e22c9090776 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:10.772229 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7cefa6-48f0-4cbf-9e0f-9e22c9090776 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:10.772624 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7cefa6-48f0-4cbf-9e0f-9e22c9090776 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:10.773765 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7cefa6-48f0-4cbf-9e0f-9e22c9090776 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:10.774137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7cefa6-48f0-4cbf-9e0f-9e22c9090776 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:10.775728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7cefa6-48f0-4cbf-9e0f-9e22c9090776 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:10.796891 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb382ee-29a3-4af6-bda9-55db5201d865 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:10.803120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 462/946] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 22:24:10 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:10.837162 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a85d2-7531-4c92-ab38-fe2f7520c0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:10.838002 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a85d2-7531-4c92-ab38-fe2f7520c0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:10.838288 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a85d2-7531-4c92-ab38-fe2f7520c0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:10.838571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a85d2-7531-4c92-ab38-fe2f7520c0d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:10.846788 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a85d2-7531-4c92-ab38-fe2f7520c0d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:10.848407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a85d2-7531-4c92-ab38-fe2f7520c0d3 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:10.865943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-481a85d2-7531-4c92-ab38-fe2f7520c0d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:24:10.865943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:10.865943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:10.865943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:10.865943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:10.865943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:10.865943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:10.865943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:10.865943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:10.865943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:10.865943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:10.865943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:10.865943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:24:10.865943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:24:10.865943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:24:10.868176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:24:10.868176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:10.868176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:10.868176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:10.868176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:10.868176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:10.868176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:10.868176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:10.868176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:10.868176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:10.868176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:10.868176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:10.868176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:10.868176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:10.868176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:10.868176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:10.868176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:10.870302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:24:10.870302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:24:10.870302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:24:10.870302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:24:10.870302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:24:10.870302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:24:10.870302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:24:10.870302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:10.874959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 485/947] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:24:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:10.875110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7cefa6-48f0-4cbf-9e0f-9e22c9090776 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:10.987522 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7cefa6-48f0-4cbf-9e0f-9e22c9090776 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:11.023261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc895171-50f6-4466-8746-986f021cc470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:11.023934 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc895171-50f6-4466-8746-986f021cc470 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:11.024220 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc895171-50f6-4466-8746-986f021cc470 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:11.024501 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc895171-50f6-4466-8746-986f021cc470 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:11.025288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc895171-50f6-4466-8746-986f021cc470 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:11.025597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc895171-50f6-4466-8746-986f021cc470 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:11.027131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc895171-50f6-4466-8746-986f021cc470 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:11.039781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 463/948] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:24:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:11.045838 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a85d2-7531-4c92-ab38-fe2f7520c0d3 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:11.120465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:11.159556 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a85d2-7531-4c92-ab38-fe2f7520c0d3 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:11.196411 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b2e0b-eb43-4e04-94b1-710c90f0074e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:11.197664 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b2e0b-eb43-4e04-94b1-710c90f0074e None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:11.198155 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b2e0b-eb43-4e04-94b1-710c90f0074e None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:11.198634 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b2e0b-eb43-4e04-94b1-710c90f0074e None admin] PATH_INFO: `/v3/projects/56da0975019c44409b53be473131143a/users/df1aeb2be41c49819b811ecfa47d4155/roles/26dbcfd7d83742b1b1cd311915a9c0a8` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:11.201104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b2e0b-eb43-4e04-94b1-710c90f0074e None admin] RBAC: Authorizing `identity:create_grant(project_id=56da0975019c44409b53be473131143a, user_id=df1aeb2be41c49819b811ecfa47d4155, role_id=26dbcfd7d83742b1b1cd311915a9c0a8)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:11.202751 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b2e0b-eb43-4e04-94b1-710c90f0074e None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:11.223085 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 486/949] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:24:11 2022] PUT /identity/v3/projects/56da0975019c44409b53be473131143a/users/df1aeb2be41c49819b811ecfa47d4155/roles/26dbcfd7d83742b1b1cd311915a9c0a8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:24:11.841464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77087a0c-23cf-4a9a-91ba-91ec1dde4b1a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:11.841978 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77087a0c-23cf-4a9a-91ba-91ec1dde4b1a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:11.842084 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77087a0c-23cf-4a9a-91ba-91ec1dde4b1a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:11.843646 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 464/950] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:11.849070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f74ca8-c2f7-447d-aa04-2e4b195b6373 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:11.850191 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f74ca8-c2f7-447d-aa04-2e4b195b6373 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:11.850808 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f74ca8-c2f7-447d-aa04-2e4b195b6373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:11.869306 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-a4f74ca8-c2f7-447d-aa04-2e4b195b6373 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:11.870811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-a4f74ca8-c2f7-447d-aa04-2e4b195b6373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:11.942374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 487/951] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:11.965753 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77087a0c-23cf-4a9a-91ba-91ec1dde4b1a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:11.983305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:12.061732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b9c4c-11cf-446d-a11a-a76d73e4fa50 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:12.061732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b9c4c-11cf-446d-a11a-a76d73e4fa50 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:12.061732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b9c4c-11cf-446d-a11a-a76d73e4fa50 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:12.061732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 488/952] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:12.070513 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4abb14-b04f-4d12-ad32-44a98cce8485 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:12.070513 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4abb14-b04f-4d12-ad32-44a98cce8485 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:12.070513 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4abb14-b04f-4d12-ad32-44a98cce8485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:12.081449 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77087a0c-23cf-4a9a-91ba-91ec1dde4b1a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:12.087234 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-4f4abb14-b04f-4d12-ad32-44a98cce8485 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:12.088651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-4f4abb14-b04f-4d12-ad32-44a98cce8485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:12.127033 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e26c2-f5e7-497c-be99-6d875c016e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:12.127033 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e26c2-f5e7-497c-be99-6d875c016e05 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:12.127033 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e26c2-f5e7-497c-be99-6d875c016e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:12.127033 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e26c2-f5e7-497c-be99-6d875c016e05 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:12.127431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e26c2-f5e7-497c-be99-6d875c016e05 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:12.127431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e26c2-f5e7-497c-be99-6d875c016e05 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:12.142667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-684e26c2-f5e7-497c-be99-6d875c016e05 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:24:12.142667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:12.142667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:24:12.142667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:24:12.142667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:24:12.142667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:24:12.142667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:24:12.142667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:12.142667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:24:12.142667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:12.142667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:24:12.142667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:12.142667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:12.142667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:12.142667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:12.142667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:12.142667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:12.143748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:12.143748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:12.143748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:12.143748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:12.143748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:12.143748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:12.143748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:24:12.143748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:24:12.143748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:24:12.143748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:24:12.143748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:12.143748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:12.143748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:12.143748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:12.143748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:12.143748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:12.145422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:12.145422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:12.145422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:12.145422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:12.145422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:12.145422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:12.145422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:12.145422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:12.145422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:12.145422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:12.145422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:24:12.145422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:12.145422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:24:12.145422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:12.145422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 465/953] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 22:24:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:12.151033 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e26c2-f5e7-497c-be99-6d875c016e05 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:12.167012 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:12.188354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 489/954] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:12.210890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4abb14-b04f-4d12-ad32-44a98cce8485 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:12.235779 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:12.266404 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e26c2-f5e7-497c-be99-6d875c016e05 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:12.321318 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ae54c-7904-404e-b168-acfab9938090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:12.322151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ae54c-7904-404e-b168-acfab9938090 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:12.322566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ae54c-7904-404e-b168-acfab9938090 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:12.322861 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ae54c-7904-404e-b168-acfab9938090 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:12.323880 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ae54c-7904-404e-b168-acfab9938090 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:12.324201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ae54c-7904-404e-b168-acfab9938090 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:12.326321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ae54c-7904-404e-b168-acfab9938090 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:12.334337 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 466/955] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 22:24:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:12.334800 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4abb14-b04f-4d12-ad32-44a98cce8485 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:12.340094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ae54c-7904-404e-b168-acfab9938090 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:12.376162 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824d3f1-ea8c-494f-b88a-4489f89723ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:12.376993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824d3f1-ea8c-494f-b88a-4489f89723ea None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:12.377339 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824d3f1-ea8c-494f-b88a-4489f89723ea None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:12.378608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824d3f1-ea8c-494f-b88a-4489f89723ea None admin] PATH_INFO: `/v3/services/compute` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:12.379192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824d3f1-ea8c-494f-b88a-4489f89723ea None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:12.381361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824d3f1-ea8c-494f-b88a-4489f89723ea None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:12.394848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-3824d3f1-ea8c-494f-b88a-4489f89723ea None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 22:24:12.394848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:12.394848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:24:12.394848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:24:12.394848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:24:12.394848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:24:12.394848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:24:12.394848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:12.394848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 22:24:12.394848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:12.394848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:24:12.394848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:12.394848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:12.394848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:12.394848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:12.394848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:12.394848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:12.396231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:12.396231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:12.396231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:12.396231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:12.396231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:12.396231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:12.396231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:24:12.396231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:24:12.396231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:24:12.396231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:24:12.396231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:12.396231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:12.396231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:12.396231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:12.396231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:12.396231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:12.397116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:12.397116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:12.397116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:12.397116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:12.397116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:12.397116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:12.397116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:12.397116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:12.397116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:12.397116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:12.397116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:24:12.397116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:12.397116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 22:24:12.397116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:12.402097 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 490/956] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:24:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:12.402996 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824d3f1-ea8c-494f-b88a-4489f89723ea None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:12.461890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:12.486915 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ae54c-7904-404e-b168-acfab9938090 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:12.518123 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824d3f1-ea8c-494f-b88a-4489f89723ea None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:12.535212 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391e50e-bb04-4483-9344-f5a4b7c4c6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:12.537301 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391e50e-bb04-4483-9344-f5a4b7c4c6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:12.537794 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391e50e-bb04-4483-9344-f5a4b7c4c6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:12.538292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391e50e-bb04-4483-9344-f5a4b7c4c6a4 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:12.539250 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391e50e-bb04-4483-9344-f5a4b7c4c6a4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:12.539774 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391e50e-bb04-4483-9344-f5a4b7c4c6a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:12.542092 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391e50e-bb04-4483-9344-f5a4b7c4c6a4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:12.549754 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 467/957] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 22:24:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:12.558099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391e50e-bb04-4483-9344-f5a4b7c4c6a4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:12.558480 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62a1dd-efaa-4e22-8e01-1b759365875b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:12.559341 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62a1dd-efaa-4e22-8e01-1b759365875b None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:12.559631 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62a1dd-efaa-4e22-8e01-1b759365875b None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:12.559921 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62a1dd-efaa-4e22-8e01-1b759365875b None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:12.560541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:12.560645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62a1dd-efaa-4e22-8e01-1b759365875b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:12.560928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62a1dd-efaa-4e22-8e01-1b759365875b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:12.562319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62a1dd-efaa-4e22-8e01-1b759365875b None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:12.572934 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 491/958] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 22:24:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:12.574035 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62a1dd-efaa-4e22-8e01-1b759365875b None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:12.680388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391e50e-bb04-4483-9344-f5a4b7c4c6a4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:12.688390 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62a1dd-efaa-4e22-8e01-1b759365875b None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:12.703918 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:12.715936 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f771b-bd8f-461c-bdea-2805436b5bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:12.716597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f771b-bd8f-461c-bdea-2805436b5bb7 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:12.716884 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f771b-bd8f-461c-bdea-2805436b5bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:12.717166 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f771b-bd8f-461c-bdea-2805436b5bb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:12.717915 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0f771b-bd8f-461c-bdea-2805436b5bb7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:12.719371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0f771b-bd8f-461c-bdea-2805436b5bb7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:12.730823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb81a04-9b46-484b-ac4c-d3e34381e79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:12.731895 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb81a04-9b46-484b-ac4c-d3e34381e79c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:12.732405 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb81a04-9b46-484b-ac4c-d3e34381e79c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:12.732887 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb81a04-9b46-484b-ac4c-d3e34381e79c None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:12.733826 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb81a04-9b46-484b-ac4c-d3e34381e79c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:12.734414 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb81a04-9b46-484b-ac4c-d3e34381e79c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:12.736228 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb81a04-9b46-484b-ac4c-d3e34381e79c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:12.740875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 468/959] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Jun 23 22:24:12 2022] POST /identity/v3/endpoints => generated 349 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:24:12.751465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 492/960] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 22:24:12 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:13.075302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271eed1-26bc-4fcd-a51c-d5ccdeebea4a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:13.075826 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271eed1-26bc-4fcd-a51c-d5ccdeebea4a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:13.075903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271eed1-26bc-4fcd-a51c-d5ccdeebea4a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:13.077489 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 469/961] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:13.082718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405a1bb-2b1e-403f-93cc-50ce78eeb789 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:13.083174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405a1bb-2b1e-403f-93cc-50ce78eeb789 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:13.083372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405a1bb-2b1e-403f-93cc-50ce78eeb789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:13.101104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-7405a1bb-2b1e-403f-93cc-50ce78eeb789 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:13.102304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7405a1bb-2b1e-403f-93cc-50ce78eeb789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:13.182717 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 493/962] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:13.206841 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271eed1-26bc-4fcd-a51c-d5ccdeebea4a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:13.271910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:13.316522 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271eed1-26bc-4fcd-a51c-d5ccdeebea4a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:13.332537 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8548011-be98-4bd8-bb8e-ab0fe5768b89 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:13.332537 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8548011-be98-4bd8-bb8e-ab0fe5768b89 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:13.332766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8548011-be98-4bd8-bb8e-ab0fe5768b89 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:13.335614 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 494/963] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:13.339701 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d28da-916f-41ba-ad7d-e98c9a73cb76 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:13.339995 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d28da-916f-41ba-ad7d-e98c9a73cb76 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:13.340279 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d28da-916f-41ba-ad7d-e98c9a73cb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:13.356201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bede881-0cf9-45a2-9cae-1917301b0c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:13.357310 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bede881-0cf9-45a2-9cae-1917301b0c31 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:13.357794 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bede881-0cf9-45a2-9cae-1917301b0c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:13.357911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-274d28da-916f-41ba-ad7d-e98c9a73cb76 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:13.358390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bede881-0cf9-45a2-9cae-1917301b0c31 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:13.359010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-274d28da-916f-41ba-ad7d-e98c9a73cb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:13.359547 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bede881-0cf9-45a2-9cae-1917301b0c31 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:13.361356 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bede881-0cf9-45a2-9cae-1917301b0c31 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:13.376986 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.sql.core [None req-4bede881-0cf9-45a2-9cae-1917301b0c31 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 22:24:13.376986 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 22:24:13.376986 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [parameters: {'user_id': '1cd2dc858e0442d9b880d38bcd58e096', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 22:24:13.376986 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80203) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 22:24:13.387609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4bede881-0cf9-45a2-9cae-1917301b0c31 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 22:24:13.387609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:13.387609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 22:24:13.387609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 22:24:13.387609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 22:24:13.387609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 22:24:13.387609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 22:24:13.387609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 22:24:13.387609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 22:24:13.387609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) Jun 23 22:24:13.387609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 22:24:13.387609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 22:24:13.387609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 22:24:13.387609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() Jun 23 22:24:13.387609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 22:24:13.388503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 22:24:13.388503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 22:24:13.388503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 22:24:13.388503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 22:24:13.388503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 22:24:13.388503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 22:24:13.388503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 22:24:13.388503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 22:24:13.388503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:13.388503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 22:24:13.388503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:13.388503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:13.388503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 22:24:13.388503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 22:24:13.388503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 23 22:24:13.388503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 23 22:24:13.388503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 22:24:13.388503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) Jun 23 22:24:13.389464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 22:24:13.389464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application yield resource Jun 23 22:24:13.389464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 22:24:13.389464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application next(self.gen) Jun 23 22:24:13.389464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 22:24:13.389464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 22:24:13.389464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 22:24:13.389464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 22:24:13.389464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 22:24:13.389464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise self.value Jun 23 22:24:13.389464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 22:24:13.389464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 22:24:13.389464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 22:24:13.389464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application session.commit() Jun 23 22:24:13.389464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 22:24:13.389464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 22:24:13.389464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 22:24:13.390350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 22:24:13.390350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 22:24:13.390350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.session.flush() Jun 23 22:24:13.390350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 22:24:13.390350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 22:24:13.390350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 22:24:13.390350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 22:24:13.390350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 22:24:13.390350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application compat.raise_( Jun 23 22:24:13.390350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 22:24:13.390350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception Jun 23 22:24:13.390350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 22:24:13.390350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 22:24:13.390350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 22:24:13.390350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 22:24:13.390350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 22:24:13.390350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 22:24:13.391291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 22:24:13.391291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 22:24:13.391291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 23 22:24:13.391291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 23 22:24:13.391291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 22:24:13.391291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 22:24:13.391291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 22:24:13.391291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 22:24:13.391291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 22:24:13.391291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 22:24:13.391291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 22:24:13.391291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 22:24:13.391291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 22:24:13.391291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 22:24:13.391291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 22:24:13.391291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception Jun 23 22:24:13.398589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 22:24:13.398589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 22:24:13.398589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 22:24:13.398589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 22:24:13.398589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 22:24:13.398589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 22:24:13.398589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 22:24:13.398589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application conn.query(q) Jun 23 22:24:13.398589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 22:24:13.398589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 22:24:13.398589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 22:24:13.398589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application result.read() Jun 23 22:24:13.398589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 22:24:13.398589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 22:24:13.398589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 22:24:13.398589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 22:24:13.400595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 22:24:13.400595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 22:24:13.400595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 22:24:13.400595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 22:24:13.400595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 22:24:13.400595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 22:24:13.400595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [parameters: {'user_id': '1cd2dc858e0442d9b880d38bcd58e096', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 22:24:13.400595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 22:24:13.400595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:13.400595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:24:13.400595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:13.400595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:13.400595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:13.400595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:13.400595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:13.402445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:13.402445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:13.402445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:13.402445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:13.402445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:13.402445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:13.402445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:13.402445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 23 22:24:13.402445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 23 22:24:13.402445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:13.402445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:13.402445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:24:13.402445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:24:13.402445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:24:13.402445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:24:13.402445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 23 22:24:13.402445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 23 22:24:13.405952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 23 22:24:13.405952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 23 22:24:13.405952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 22:24:13.405952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 22:24:13.405952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 22:24:13.405952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:13.405952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 470/964] 10.222.0.43 () {60 vars in 1131 bytes} [Thu Jun 23 22:24:13 2022] POST /identity/v3/users => generated 160 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:13.405952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bede881-0cf9-45a2-9cae-1917301b0c31 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:13.457987 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 495/965] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:13.482192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d28da-916f-41ba-ad7d-e98c9a73cb76 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:13.526764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:13.541553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bede881-0cf9-45a2-9cae-1917301b0c31 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:13.573609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:13.591179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e283a33-9dc8-4fc8-ac8f-580786e840e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:13.592518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e283a33-9dc8-4fc8-ac8f-580786e840e2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:13.593031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e283a33-9dc8-4fc8-ac8f-580786e840e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:13.593895 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e283a33-9dc8-4fc8-ac8f-580786e840e2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:13.603930 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e283a33-9dc8-4fc8-ac8f-580786e840e2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:13.607087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e283a33-9dc8-4fc8-ac8f-580786e840e2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:13.614511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d28da-916f-41ba-ad7d-e98c9a73cb76 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:13.626362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-2e283a33-9dc8-4fc8-ac8f-580786e840e2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 22:24:13.626362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:13.626362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:13.626362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:13.626362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:13.626362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:13.626362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:13.626362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:13.626362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:13.626362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:13.626362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:13.626362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:13.626362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:24:13.626362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:24:13.626362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:24:13.626362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:24:13.627325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:13.627325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:13.627325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:24:13.627325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:24:13.627325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:24:13.627325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:24:13.627325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:13.627325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:13.627325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:13.627325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:13.627325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:13.627325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:13.627325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:13.627325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:13.627325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:13.627325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:13.627325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:13.627325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:13.628282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:13.628282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:13.628282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:24:13.628282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:24:13.628282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:24:13.628282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:24:13.628282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:24:13.628282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:24:13.628282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 22:24:13.628282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:13.630985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 471/966] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:24:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:13.634629 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e283a33-9dc8-4fc8-ac8f-580786e840e2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:13.663002 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311c15a-8795-4e1a-bc36-7b65900b8610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:13.663827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311c15a-8795-4e1a-bc36-7b65900b8610 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:13.664112 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311c15a-8795-4e1a-bc36-7b65900b8610 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:13.664391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311c15a-8795-4e1a-bc36-7b65900b8610 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:13.678310 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311c15a-8795-4e1a-bc36-7b65900b8610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:24:13.688809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-5311c15a-8795-4e1a-bc36-7b65900b8610 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:24:13.688809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:13.688809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:13.688809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:13.688809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:13.688809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:13.688809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:13.688809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:13.688809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:13.688809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:13.688809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:13.688809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:13.688809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:24:13.688809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:24:13.688809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:24:13.688809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:24:13.689748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:24:13.689748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:24:13.689748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:24:13.689748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:24:13.689748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:13.689748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:13.689748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:13.689748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:13.689748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:13.689748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:13.689748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:13.689748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:13.689748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:13.689748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:13.689748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:13.689748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:13.690610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:13.690610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:13.690610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:13.690610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:13.690610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:24:13.690610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:24:13.690610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:24:13.690610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:24:13.690610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:24:13.690610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:24:13.690610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:24:13.690610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:13.695125 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 496/967] 10.222.0.43 () {56 vars in 1120 bytes} [Thu Jun 23 22:24:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:13.696536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311c15a-8795-4e1a-bc36-7b65900b8610 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:13.767675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e283a33-9dc8-4fc8-ac8f-580786e840e2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:13.815888 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec482fb-6664-4c7e-80e0-288885a52b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:13.816596 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec482fb-6664-4c7e-80e0-288885a52b5f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:13.817145 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec482fb-6664-4c7e-80e0-288885a52b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:13.817433 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec482fb-6664-4c7e-80e0-288885a52b5f None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:13.819285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec482fb-6664-4c7e-80e0-288885a52b5f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:13.819603 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec482fb-6664-4c7e-80e0-288885a52b5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:13.821729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec482fb-6664-4c7e-80e0-288885a52b5f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:13.823251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:13.831700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311c15a-8795-4e1a-bc36-7b65900b8610 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:13.875404 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 472/968] 10.222.0.43 () {56 vars in 1101 bytes} [Thu Jun 23 22:24:13 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:13.878842 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3b649-c295-4a56-94b3-8b621b76fd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:13.879644 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3b649-c295-4a56-94b3-8b621b76fd0d None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:13.879935 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3b649-c295-4a56-94b3-8b621b76fd0d None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:13.880259 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3b649-c295-4a56-94b3-8b621b76fd0d None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:13.881106 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3b649-c295-4a56-94b3-8b621b76fd0d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:13.881423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3b649-c295-4a56-94b3-8b621b76fd0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:13.882833 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3b649-c295-4a56-94b3-8b621b76fd0d None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:13.893545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 497/969] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Jun 23 22:24:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:13.903902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec482fb-6664-4c7e-80e0-288885a52b5f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:13.960200 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:14.026047 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec482fb-6664-4c7e-80e0-288885a52b5f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:14.065902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5c073-1287-4631-8c48-e4611177c228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:14.066701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5c073-1287-4631-8c48-e4611177c228 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:14.067100 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5c073-1287-4631-8c48-e4611177c228 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:14.067347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5c073-1287-4631-8c48-e4611177c228 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:14.078437 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5c073-1287-4631-8c48-e4611177c228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:14.080000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5c073-1287-4631-8c48-e4611177c228 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:14.097749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d5c5c073-1287-4631-8c48-e4611177c228 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:14.097749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:14.097749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:14.097749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:14.097749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:14.097749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:14.097749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:14.097749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:14.097749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:14.097749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:14.097749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:14.097749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:14.097749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:24:14.097749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:24:14.097749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:24:14.098685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:24:14.098685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:14.098685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:14.098685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:14.098685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:14.098685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:14.098685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:14.098685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:14.098685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:14.098685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:14.098685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:14.098685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:14.098685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:14.098685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:14.098685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:14.098685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:14.098685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:14.100892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:24:14.100892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:24:14.100892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:14.100892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:14.100892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 473/970] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:14.105332 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3b649-c295-4a56-94b3-8b621b76fd0d None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:14.218007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3b649-c295-4a56-94b3-8b621b76fd0d None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:14.251008 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e47fe7-7198-4ad3-8048-5855645a22f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:14.251673 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e47fe7-7198-4ad3-8048-5855645a22f3 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:14.251958 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e47fe7-7198-4ad3-8048-5855645a22f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:14.252236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e47fe7-7198-4ad3-8048-5855645a22f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:14.252957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e47fe7-7198-4ad3-8048-5855645a22f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:14.253250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e47fe7-7198-4ad3-8048-5855645a22f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:14.254603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e47fe7-7198-4ad3-8048-5855645a22f3 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:14.268111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 498/971] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:14.273926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5c073-1287-4631-8c48-e4611177c228 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:14.365927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:14.375986 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5c073-1287-4631-8c48-e4611177c228 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:14.409834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ab45e-4de9-42b5-96e3-45196b023247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:14.410565 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ab45e-4de9-42b5-96e3-45196b023247 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:14.410852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ab45e-4de9-42b5-96e3-45196b023247 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:14.411162 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ab45e-4de9-42b5-96e3-45196b023247 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:14.420510 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ab45e-4de9-42b5-96e3-45196b023247 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:14.421951 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ab45e-4de9-42b5-96e3-45196b023247 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:14.436998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ef8ab45e-4de9-42b5-96e3-45196b023247 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:24:14.436998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:14.436998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:14.436998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:14.436998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:14.436998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:14.436998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:14.436998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:14.436998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:14.436998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:14.436998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:14.436998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:14.436998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:24:14.436998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:24:14.436998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:24:14.436998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:24:14.437982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:14.437982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:14.437982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:24:14.437982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:24:14.437982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:24:14.437982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:24:14.437982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:14.437982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:14.437982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:14.437982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:14.437982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:14.437982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:14.437982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:14.437982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:14.437982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:14.437982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:14.437982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:14.437982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:14.438952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:14.438952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:14.438952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:24:14.438952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:24:14.438952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:24:14.438952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:24:14.438952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:24:14.438952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:24:14.438952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 22:24:14.438952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:14.440406 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 474/972] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 22:24:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:14.444505 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e47fe7-7198-4ad3-8048-5855645a22f3 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:14.463467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:14.550199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e47fe7-7198-4ad3-8048-5855645a22f3 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:14.585386 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd171a70-9c94-4d0e-8824-0b928d736695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:14.586067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd171a70-9c94-4d0e-8824-0b928d736695 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:14.586361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd171a70-9c94-4d0e-8824-0b928d736695 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:14.586647 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd171a70-9c94-4d0e-8824-0b928d736695 None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:14.587518 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd171a70-9c94-4d0e-8824-0b928d736695 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:14.587838 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd171a70-9c94-4d0e-8824-0b928d736695 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:14.589272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd171a70-9c94-4d0e-8824-0b928d736695 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:14.611351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 499/973] 10.222.0.43 () {56 vars in 1149 bytes} [Thu Jun 23 22:24:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:14.612921 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ab45e-4de9-42b5-96e3-45196b023247 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:14.714348 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ab45e-4de9-42b5-96e3-45196b023247 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:14.748775 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f8f0a-8d80-46c8-b187-bbc738a1566a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:14.749422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f8f0a-8d80-46c8-b187-bbc738a1566a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:14.749706 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f8f0a-8d80-46c8-b187-bbc738a1566a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:14.749981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f8f0a-8d80-46c8-b187-bbc738a1566a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:14.771244 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:14.818627 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f8f0a-8d80-46c8-b187-bbc738a1566a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:14.820227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f8f0a-8d80-46c8-b187-bbc738a1566a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:14.852757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8f4b8-0a76-4de5-a53f-a213026fac24 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:14.853041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8f4b8-0a76-4de5-a53f-a213026fac24 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:14.853320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8f4b8-0a76-4de5-a53f-a213026fac24 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:14.854845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 500/974] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:24:14.859545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c27eb0-2f0a-45ae-afa7-21b02341e64b None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:14.859837 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c27eb0-2f0a-45ae-afa7-21b02341e64b None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:14.860121 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c27eb0-2f0a-45ae-afa7-21b02341e64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:14.877066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-93c27eb0-2f0a-45ae-afa7-21b02341e64b None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:14.878218 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-93c27eb0-2f0a-45ae-afa7-21b02341e64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:14.938179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 475/975] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:24:14 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 22:24:14.938179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff57fe-a801-404c-95e1-362b7b66d815 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:14.938179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff57fe-a801-404c-95e1-362b7b66d815 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:14.938179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff57fe-a801-404c-95e1-362b7b66d815 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:14.938179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 476/976] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:14.939560 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff57fe-a801-404c-95e1-362b7b66d815 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:14.952503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 501/977] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:14.953783 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da27c8d-0eb5-4a3d-8363-f07e794188b4 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:14.954078 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da27c8d-0eb5-4a3d-8363-f07e794188b4 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:14.954355 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da27c8d-0eb5-4a3d-8363-f07e794188b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:14.960998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:14.972654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-1da27c8d-0eb5-4a3d-8363-f07e794188b4 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:14.973867 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1da27c8d-0eb5-4a3d-8363-f07e794188b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:14.995902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:15.040806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 502/978] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:15.041837 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da27c8d-0eb5-4a3d-8363-f07e794188b4 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:15.043989 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff57fe-a801-404c-95e1-362b7b66d815 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:15.079289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e3bc2-31d1-4a68-bee3-6cdf9fc06d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:15.080195 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e3bc2-31d1-4a68-bee3-6cdf9fc06d44 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:15.080484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e3bc2-31d1-4a68-bee3-6cdf9fc06d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:15.080770 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e3bc2-31d1-4a68-bee3-6cdf9fc06d44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:15.091507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e3bc2-31d1-4a68-bee3-6cdf9fc06d44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:15.092940 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e3bc2-31d1-4a68-bee3-6cdf9fc06d44 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:15.110201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-107e3bc2-31d1-4a68-bee3-6cdf9fc06d44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:15.110201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:15.110201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:15.110201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:15.110201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:15.110201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:15.110201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:15.110201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:15.110201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:15.110201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:15.110201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:15.110201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:15.110201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:24:15.110201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:24:15.110201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:24:15.111261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:24:15.111261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:15.111261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:15.111261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:15.111261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:15.111261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:15.111261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:15.111261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:15.111261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:15.111261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:15.111261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:15.111261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:15.111261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:15.111261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:15.111261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:15.111261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:15.111261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:15.112729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:24:15.112729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:24:15.112729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:15.112729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:15.112729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 477/979] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:15.112729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e3bc2-31d1-4a68-bee3-6cdf9fc06d44 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:15.153922 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da27c8d-0eb5-4a3d-8363-f07e794188b4 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:15.188294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fa705-8cd6-4102-9082-1aacd502a8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:15.188967 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fa705-8cd6-4102-9082-1aacd502a8b6 None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:15.189258 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fa705-8cd6-4102-9082-1aacd502a8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:15.189543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fa705-8cd6-4102-9082-1aacd502a8b6 None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:15.190269 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7fa705-8cd6-4102-9082-1aacd502a8b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:15.191682 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7fa705-8cd6-4102-9082-1aacd502a8b6 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:15.193767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:15.206137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 503/980] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 22:24:15 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:24:15.206957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fa705-8cd6-4102-9082-1aacd502a8b6 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:15.235340 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e3bc2-31d1-4a68-bee3-6cdf9fc06d44 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:15.238990 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:15.283923 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51540910-a5f7-4bd0-9156-61279f4a5a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:15.284817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51540910-a5f7-4bd0-9156-61279f4a5a32 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:15.285117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51540910-a5f7-4bd0-9156-61279f4a5a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:15.285403 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51540910-a5f7-4bd0-9156-61279f4a5a32 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:15.286289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51540910-a5f7-4bd0-9156-61279f4a5a32 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:15.289093 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51540910-a5f7-4bd0-9156-61279f4a5a32 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:15.302439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-51540910-a5f7-4bd0-9156-61279f4a5a32 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:24:15.302439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:15.302439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:24:15.302439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:24:15.302439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:24:15.302439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:24:15.302439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:24:15.302439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:15.302439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:24:15.302439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:15.302439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:24:15.302439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:15.302439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:15.302439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:15.302439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:15.302439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:15.302439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:15.303736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:15.303736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:15.303736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:15.303736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:15.303736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:15.303736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:15.303736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:24:15.303736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:24:15.303736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:24:15.303736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:24:15.303736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:15.303736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:15.303736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:15.303736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:15.303736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:15.303736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:15.310078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:15.310078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:15.310078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:15.310078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:15.310078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:15.310078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:15.310078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:15.310078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:15.310078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:15.310078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:15.310078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:24:15.310078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:15.310078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:24:15.310078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:15.310078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 478/981] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 22:24:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:15.311099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51540910-a5f7-4bd0-9156-61279f4a5a32 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:15.332071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fa705-8cd6-4102-9082-1aacd502a8b6 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:15.375447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16188f2-0f15-42a2-ae05-273fa8e7370c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:15.376920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16188f2-0f15-42a2-ae05-273fa8e7370c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:15.376920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16188f2-0f15-42a2-ae05-273fa8e7370c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:15.376920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16188f2-0f15-42a2-ae05-273fa8e7370c None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:15.377471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16188f2-0f15-42a2-ae05-273fa8e7370c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:15.377794 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16188f2-0f15-42a2-ae05-273fa8e7370c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:15.379721 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16188f2-0f15-42a2-ae05-273fa8e7370c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:15.396926 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 504/982] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:15.402915 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16188f2-0f15-42a2-ae05-273fa8e7370c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:15.432902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51540910-a5f7-4bd0-9156-61279f4a5a32 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:15.469637 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:15.470577 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c34ce0-2d09-40e6-944e-75a14e2f0d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:15.471498 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c34ce0-2d09-40e6-944e-75a14e2f0d80 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:15.471911 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c34ce0-2d09-40e6-944e-75a14e2f0d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:15.472331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c34ce0-2d09-40e6-944e-75a14e2f0d80 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:15.473159 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c34ce0-2d09-40e6-944e-75a14e2f0d80 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:15.473531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c34ce0-2d09-40e6-944e-75a14e2f0d80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:15.475543 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c34ce0-2d09-40e6-944e-75a14e2f0d80 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:15.481223 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 479/983] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 22:24:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:15.486843 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c34ce0-2d09-40e6-944e-75a14e2f0d80 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:15.516620 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:15.524310 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16188f2-0f15-42a2-ae05-273fa8e7370c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:15.562052 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbd366-ed6e-4100-8ec1-8b4c995fd856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:15.563124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbd366-ed6e-4100-8ec1-8b4c995fd856 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:15.563585 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbd366-ed6e-4100-8ec1-8b4c995fd856 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:15.564174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbd366-ed6e-4100-8ec1-8b4c995fd856 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:15.572254 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cbd366-ed6e-4100-8ec1-8b4c995fd856 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:15.573868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cbd366-ed6e-4100-8ec1-8b4c995fd856 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:15.588277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-00cbd366-ed6e-4100-8ec1-8b4c995fd856 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:24:15.588277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:15.588277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:15.588277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:15.588277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:15.588277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:15.588277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:15.588277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:15.588277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:15.588277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:15.588277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:15.588277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:15.588277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:24:15.588277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:24:15.588277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:24:15.589211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:24:15.589211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:15.589211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:15.589211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:15.589211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:15.589211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:15.589211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:15.589211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:15.589211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:15.589211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:15.589211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:15.589211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:15.589211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:15.589211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:15.589211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:15.589211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:15.589211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:15.590099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:24:15.590099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:24:15.590099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:24:15.590099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:24:15.590099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:24:15.590099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:24:15.590099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:24:15.590099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:15.594298 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 505/984] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:24:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:15.595164 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbd366-ed6e-4100-8ec1-8b4c995fd856 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:15.600718 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c34ce0-2d09-40e6-944e-75a14e2f0d80 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:15.637419 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285d674-aa14-4af4-97e9-ebd47e1bda03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:15.637774 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285d674-aa14-4af4-97e9-ebd47e1bda03 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:15.638069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285d674-aa14-4af4-97e9-ebd47e1bda03 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:15.638359 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285d674-aa14-4af4-97e9-ebd47e1bda03 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:15.639106 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285d674-aa14-4af4-97e9-ebd47e1bda03 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:15.639428 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285d674-aa14-4af4-97e9-ebd47e1bda03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:15.640793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285d674-aa14-4af4-97e9-ebd47e1bda03 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:15.646016 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 480/985] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 22:24:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:15.651897 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285d674-aa14-4af4-97e9-ebd47e1bda03 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:15.709676 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbd366-ed6e-4100-8ec1-8b4c995fd856 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:15.726065 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:15.747827 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:15.751384 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651daea2-e341-400a-ac23-ede1735aaa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:15.752064 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651daea2-e341-400a-ac23-ede1735aaa61 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:15.752392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651daea2-e341-400a-ac23-ede1735aaa61 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:15.752648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651daea2-e341-400a-ac23-ede1735aaa61 None admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:15.753440 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651daea2-e341-400a-ac23-ede1735aaa61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:15.753759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651daea2-e341-400a-ac23-ede1735aaa61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:15.755486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651daea2-e341-400a-ac23-ede1735aaa61 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:15.757716 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285d674-aa14-4af4-97e9-ebd47e1bda03 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:15.768769 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 506/986] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:24:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:15.775793 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651daea2-e341-400a-ac23-ede1735aaa61 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:15.807352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02029c3a-c6cb-4bd6-ba04-3d2938e93c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:15.808019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02029c3a-c6cb-4bd6-ba04-3d2938e93c10 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:15.808308 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02029c3a-c6cb-4bd6-ba04-3d2938e93c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:15.808580 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02029c3a-c6cb-4bd6-ba04-3d2938e93c10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:15.809377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02029c3a-c6cb-4bd6-ba04-3d2938e93c10 None admin] RBAC: Adding query filter params (interface=admin, service_id=b04f1788e9734fe4a3e334440f7889a0, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:15.809696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02029c3a-c6cb-4bd6-ba04-3d2938e93c10 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:15.811082 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02029c3a-c6cb-4bd6-ba04-3d2938e93c10 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:15.816147 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 481/987] 10.222.0.43 () {56 vars in 1249 bytes} [Thu Jun 23 22:24:15 2022] GET /identity/v3/endpoints?service_id=b04f1788e9734fe4a3e334440f7889a0&interface=admin®ion_id=RegionOne => generated 193 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:15.821617 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02029c3a-c6cb-4bd6-ba04-3d2938e93c10 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:15.884559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651daea2-e341-400a-ac23-ede1735aaa61 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:15.919801 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93537301-a104-46a9-a0f8-5e44e7058328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:15.920708 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93537301-a104-46a9-a0f8-5e44e7058328 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:15.921173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93537301-a104-46a9-a0f8-5e44e7058328 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:15.921621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93537301-a104-46a9-a0f8-5e44e7058328 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:15.922601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93537301-a104-46a9-a0f8-5e44e7058328 None admin] RBAC: Adding query filter params (role.id=26dbcfd7d83742b1b1cd311915a9c0a8, scope.project.id=56da0975019c44409b53be473131143a, user.id=df1aeb2be41c49819b811ecfa47d4155) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:15.923103 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93537301-a104-46a9-a0f8-5e44e7058328 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:15.924721 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93537301-a104-46a9-a0f8-5e44e7058328 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:15.926078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02029c3a-c6cb-4bd6-ba04-3d2938e93c10 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:15.931960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 507/988] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:24:15 2022] GET /identity/v3/role_assignments?user.id=df1aeb2be41c49819b811ecfa47d4155&scope.project.id=56da0975019c44409b53be473131143a&role.id=26dbcfd7d83742b1b1cd311915a9c0a8 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:15.975429 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3eac7-07e1-4be5-a8d9-bca6b2b4c905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:15.976162 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3eac7-07e1-4be5-a8d9-bca6b2b4c905 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:15.976451 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3eac7-07e1-4be5-a8d9-bca6b2b4c905 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:15.976730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3eac7-07e1-4be5-a8d9-bca6b2b4c905 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:15.977423 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3eac7-07e1-4be5-a8d9-bca6b2b4c905 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:15.978801 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3eac7-07e1-4be5-a8d9-bca6b2b4c905 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:15.986205 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 482/989] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Jun 23 22:24:15 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:24:17.278189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149410ff-554d-4c10-a65e-a1b2e7029270 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:17.278701 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149410ff-554d-4c10-a65e-a1b2e7029270 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:17.278780 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149410ff-554d-4c10-a65e-a1b2e7029270 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:17.280295 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 508/990] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:17.285112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0a117-212b-408d-93e6-43eb859d1e8e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:17.285507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0a117-212b-408d-93e6-43eb859d1e8e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:17.285668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0a117-212b-408d-93e6-43eb859d1e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:17.302836 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d8d0a117-212b-408d-93e6-43eb859d1e8e None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:17.304344 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d8d0a117-212b-408d-93e6-43eb859d1e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:17.317957 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:17.370285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 483/991] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:17.393008 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149410ff-554d-4c10-a65e-a1b2e7029270 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:17.436296 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:17.500024 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149410ff-554d-4c10-a65e-a1b2e7029270 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:17.534601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a3f60-744e-4197-9026-3ed7859e0ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:17.535436 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a3f60-744e-4197-9026-3ed7859e0ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:17.535729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a3f60-744e-4197-9026-3ed7859e0ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:17.536012 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a3f60-744e-4197-9026-3ed7859e0ab8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:17.536722 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a3f60-744e-4197-9026-3ed7859e0ab8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:17.538269 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a3f60-744e-4197-9026-3ed7859e0ab8 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:17.550015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-260a3f60-744e-4197-9026-3ed7859e0ab8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 22:24:17.550015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:17.550015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:24:17.550015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:24:17.550015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:24:17.550015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:24:17.550015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:24:17.550015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:17.550015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 22:24:17.550015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:17.550015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:24:17.550015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:17.550015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:17.550015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:17.550015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:17.550015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:17.550015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:17.551261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:17.551261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:17.551261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:17.551261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:17.551261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:17.551261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:17.551261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:24:17.551261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:24:17.551261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:24:17.551261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:24:17.551261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:17.551261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:17.551261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:17.551261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:17.551261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:17.551261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:17.552997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:17.552997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:17.552997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:17.552997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:17.552997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:17.552997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:17.552997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:17.552997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:17.552997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:17.552997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:17.552997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:24:17.552997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:17.552997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 22:24:17.552997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:17.555279 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 509/992] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:24:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:17.557879 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0a117-212b-408d-93e6-43eb859d1e8e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:17.661290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d0a117-212b-408d-93e6-43eb859d1e8e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:17.696778 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725eaa8-9bcb-4f05-9cfb-dccb0185c965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:17.697449 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725eaa8-9bcb-4f05-9cfb-dccb0185c965 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:17.697736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725eaa8-9bcb-4f05-9cfb-dccb0185c965 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:17.698019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725eaa8-9bcb-4f05-9cfb-dccb0185c965 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:17.698728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725eaa8-9bcb-4f05-9cfb-dccb0185c965 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:17.699071 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725eaa8-9bcb-4f05-9cfb-dccb0185c965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:17.700439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725eaa8-9bcb-4f05-9cfb-dccb0185c965 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:17.710074 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 484/993] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 22:24:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:17.711168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a3f60-744e-4197-9026-3ed7859e0ab8 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:17.814914 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a3f60-744e-4197-9026-3ed7859e0ab8 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:17.816477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:17.851389 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058ec64-ab64-4809-ac2f-7a8c61bd4d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:17.852066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058ec64-ab64-4809-ac2f-7a8c61bd4d9b None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:17.852356 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058ec64-ab64-4809-ac2f-7a8c61bd4d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:17.852639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058ec64-ab64-4809-ac2f-7a8c61bd4d9b None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:17.853346 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058ec64-ab64-4809-ac2f-7a8c61bd4d9b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:17.853656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058ec64-ab64-4809-ac2f-7a8c61bd4d9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:17.855112 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058ec64-ab64-4809-ac2f-7a8c61bd4d9b None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:17.864859 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 510/994] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 22:24:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:17.866111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725eaa8-9bcb-4f05-9cfb-dccb0185c965 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:17.883635 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:17.970086 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725eaa8-9bcb-4f05-9cfb-dccb0185c965 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:18.003306 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b64d26-6e84-4038-8b7e-3b3d1a504cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:18.003977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b64d26-6e84-4038-8b7e-3b3d1a504cbd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:18.004264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b64d26-6e84-4038-8b7e-3b3d1a504cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:18.004547 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b64d26-6e84-4038-8b7e-3b3d1a504cbd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:18.005889 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b64d26-6e84-4038-8b7e-3b3d1a504cbd None admin] RBAC: Adding query filter params (interface=public, service_id=9c934bcbc7564d419a6c803db7a16826, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:18.006212 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b64d26-6e84-4038-8b7e-3b3d1a504cbd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:18.007604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b64d26-6e84-4038-8b7e-3b3d1a504cbd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:18.017312 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 485/995] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Jun 23 22:24:17 2022] GET /identity/v3/endpoints?service_id=9c934bcbc7564d419a6c803db7a16826&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:18.018279 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058ec64-ab64-4809-ac2f-7a8c61bd4d9b None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:18.029970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ca06e-2eb1-472e-8459-964b3bed2876 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:18.030257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ca06e-2eb1-472e-8459-964b3bed2876 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:18.030542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ca06e-2eb1-472e-8459-964b3bed2876 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:18.032089 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 486/996] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:24:18.036811 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7138036-2a85-4200-a62a-198725337d50 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:18.037333 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7138036-2a85-4200-a62a-198725337d50 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:18.037624 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7138036-2a85-4200-a62a-198725337d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:18.054469 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a7138036-2a85-4200-a62a-198725337d50 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:18.055694 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a7138036-2a85-4200-a62a-198725337d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:18.126965 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 487/997] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:18.127705 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058ec64-ab64-4809-ac2f-7a8c61bd4d9b None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:18.127789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006df10-2cc7-44b3-8cf9-910ff22825f5 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:18.127789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006df10-2cc7-44b3-8cf9-910ff22825f5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:18.127789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006df10-2cc7-44b3-8cf9-910ff22825f5 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:18.127789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 488/998] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:18.131315 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1fe53-00f7-4a03-b123-38b44af5d6cc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:18.131605 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1fe53-00f7-4a03-b123-38b44af5d6cc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:18.131894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1fe53-00f7-4a03-b123-38b44af5d6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:18.149959 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cac1fe53-00f7-4a03-b123-38b44af5d6cc None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:18.151184 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cac1fe53-00f7-4a03-b123-38b44af5d6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:18.153602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:18.163748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36771c76-aeac-4b57-9571-979167d3dac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:18.164498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36771c76-aeac-4b57-9571-979167d3dac5 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:18.164782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36771c76-aeac-4b57-9571-979167d3dac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:18.165075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36771c76-aeac-4b57-9571-979167d3dac5 None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:18.165737 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36771c76-aeac-4b57-9571-979167d3dac5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:18.167138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36771c76-aeac-4b57-9571-979167d3dac5 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:18.173245 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 511/999] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Jun 23 22:24:18 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:24:18.174007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36771c76-aeac-4b57-9571-979167d3dac5 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:18.222682 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:18.238952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 489/1000] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:18.249454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1fe53-00f7-4a03-b123-38b44af5d6cc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:18.315649 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36771c76-aeac-4b57-9571-979167d3dac5 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:18.353806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771578f-1a23-4249-84e1-e7ec1b7b8961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:18.354620 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771578f-1a23-4249-84e1-e7ec1b7b8961 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:18.354912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771578f-1a23-4249-84e1-e7ec1b7b8961 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:18.355227 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771578f-1a23-4249-84e1-e7ec1b7b8961 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:18.355878 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771578f-1a23-4249-84e1-e7ec1b7b8961 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:18.357248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771578f-1a23-4249-84e1-e7ec1b7b8961 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:18.366253 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1fe53-00f7-4a03-b123-38b44af5d6cc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:18.369746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-6771578f-1a23-4249-84e1-e7ec1b7b8961 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:24:18.369746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:18.369746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:24:18.369746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:24:18.369746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:24:18.369746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:24:18.369746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:24:18.369746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:18.369746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:24:18.369746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:18.369746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:24:18.369746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:18.369746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:18.369746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:18.369746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:18.369746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:18.369746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:18.370735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:18.370735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:18.370735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:18.370735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:18.370735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:18.370735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:18.370735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:24:18.370735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:24:18.370735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:24:18.370735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:24:18.370735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:18.370735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:18.370735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:18.370735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:18.370735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:18.370735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:18.372104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:18.372104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:18.372104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:18.372104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:18.372104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:18.372104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:18.372104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:18.372104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:18.372104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:18.372104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:18.372104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:24:18.372104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:18.372104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 22:24:18.372104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:18.372104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 512/1001] 10.222.0.43 () {56 vars in 1126 bytes} [Thu Jun 23 22:24:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:18.376771 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771578f-1a23-4249-84e1-e7ec1b7b8961 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:18.408769 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23f955-bc6a-4a40-a295-edf52c9dc8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:18.409563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23f955-bc6a-4a40-a295-edf52c9dc8fc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:18.409879 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23f955-bc6a-4a40-a295-edf52c9dc8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:18.410149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23f955-bc6a-4a40-a295-edf52c9dc8fc None admin] PATH_INFO: `/v3/services/image` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:18.410776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23f955-bc6a-4a40-a295-edf52c9dc8fc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:18.413000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23f955-bc6a-4a40-a295-edf52c9dc8fc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:18.425533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-8f23f955-bc6a-4a40-a295-edf52c9dc8fc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 22:24:18.425533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:18.425533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:24:18.425533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:24:18.425533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:24:18.425533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:24:18.425533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:24:18.425533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:18.425533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 22:24:18.425533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:18.425533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:24:18.425533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:18.425533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:18.425533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:18.425533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:18.425533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:18.425533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:18.426582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:18.426582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:18.426582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:18.426582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:18.426582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:18.426582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:18.426582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:24:18.426582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:24:18.426582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:24:18.426582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:24:18.426582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:18.426582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:18.426582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:18.426582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:18.426582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:18.426582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:18.428111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:18.428111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:18.428111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:18.428111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:18.428111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:18.428111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:18.428111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:18.428111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:18.428111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:18.428111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:18.428111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:24:18.428111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:18.428111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 22:24:18.428111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:18.428111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 490/1002] 10.222.0.43 () {56 vars in 1105 bytes} [Thu Jun 23 22:24:18 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:18.432338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23f955-bc6a-4a40-a295-edf52c9dc8fc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:18.435393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:18.480269 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:18.487183 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771578f-1a23-4249-84e1-e7ec1b7b8961 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:18.520782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24402dc-31f2-4b1a-8587-049b52363ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:18.521539 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24402dc-31f2-4b1a-8587-049b52363ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:18.521827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24402dc-31f2-4b1a-8587-049b52363ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:18.522107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24402dc-31f2-4b1a-8587-049b52363ed0 None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:18.522803 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24402dc-31f2-4b1a-8587-049b52363ed0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:18.523396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24402dc-31f2-4b1a-8587-049b52363ed0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:18.524757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24402dc-31f2-4b1a-8587-049b52363ed0 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:18.534162 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 513/1003] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 22:24:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:18.535069 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24402dc-31f2-4b1a-8587-049b52363ed0 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:18.548027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23f955-bc6a-4a40-a295-edf52c9dc8fc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:18.584011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32bd44-5432-47e8-98cc-4251c7418290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:18.584772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32bd44-5432-47e8-98cc-4251c7418290 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:18.585068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32bd44-5432-47e8-98cc-4251c7418290 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:18.585353 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32bd44-5432-47e8-98cc-4251c7418290 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:18.586060 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32bd44-5432-47e8-98cc-4251c7418290 None admin] RBAC: Adding query filter params (name=image) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:18.586373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32bd44-5432-47e8-98cc-4251c7418290 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:18.587898 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f32bd44-5432-47e8-98cc-4251c7418290 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:18.597594 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 491/1004] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:18.598582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32bd44-5432-47e8-98cc-4251c7418290 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:18.639833 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24402dc-31f2-4b1a-8587-049b52363ed0 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:18.670144 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:18.681062 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03edfd-da2e-4d12-ba6f-50e35cd44b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:18.681742 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03edfd-da2e-4d12-ba6f-50e35cd44b2b None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:18.682027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03edfd-da2e-4d12-ba6f-50e35cd44b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:18.682303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03edfd-da2e-4d12-ba6f-50e35cd44b2b None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:18.683454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03edfd-da2e-4d12-ba6f-50e35cd44b2b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:18.683765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03edfd-da2e-4d12-ba6f-50e35cd44b2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:18.685489 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03edfd-da2e-4d12-ba6f-50e35cd44b2b None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:18.695874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 514/1005] 10.222.0.43 () {56 vars in 1122 bytes} [Thu Jun 23 22:24:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:18.696817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03edfd-da2e-4d12-ba6f-50e35cd44b2b None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:18.714563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32bd44-5432-47e8-98cc-4251c7418290 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:18.722176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:18.750268 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f836c-a7ec-49d1-8451-619c7f42594c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:18.750863 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f836c-a7ec-49d1-8451-619c7f42594c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:18.751182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f836c-a7ec-49d1-8451-619c7f42594c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:18.751465 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f836c-a7ec-49d1-8451-619c7f42594c None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:18.752162 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f836c-a7ec-49d1-8451-619c7f42594c None admin] RBAC: Adding query filter params (type=image) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:18.752477 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f836c-a7ec-49d1-8451-619c7f42594c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:18.753838 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f836c-a7ec-49d1-8451-619c7f42594c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:18.766967 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 492/1006] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:18 2022] GET /identity/v3/services?type=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:18.812401 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03edfd-da2e-4d12-ba6f-50e35cd44b2b None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:18.850425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b0c97-d8f8-47a3-b1bc-254cc6098fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:18.851124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b0c97-d8f8-47a3-b1bc-254cc6098fec None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:18.851410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b0c97-d8f8-47a3-b1bc-254cc6098fec None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:18.851686 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b0c97-d8f8-47a3-b1bc-254cc6098fec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:18.852440 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b0c97-d8f8-47a3-b1bc-254cc6098fec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:18.853818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b0c97-d8f8-47a3-b1bc-254cc6098fec None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:18.871467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 515/1007] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Jun 23 22:24:18 2022] POST /identity/v3/endpoints => generated 325 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:24:20.245248 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e669c73-b767-45cb-975e-1bc12c048bfd None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:20.245783 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e669c73-b767-45cb-975e-1bc12c048bfd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:20.245874 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e669c73-b767-45cb-975e-1bc12c048bfd None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:20.247369 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 493/1008] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:20.252322 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d7750-c72d-4450-a165-d2b2fb05e72c None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:20.252693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d7750-c72d-4450-a165-d2b2fb05e72c None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:20.252898 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d7750-c72d-4450-a165-d2b2fb05e72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:20.270248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-989d7750-c72d-4450-a165-d2b2fb05e72c None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:20.271523 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-989d7750-c72d-4450-a165-d2b2fb05e72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:20.346495 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 516/1009] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:20.370468 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e669c73-b767-45cb-975e-1bc12c048bfd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:20.474506 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e669c73-b767-45cb-975e-1bc12c048bfd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:20.506808 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:20.514345 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724694c4-3dbe-478a-8427-6d57687ff696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:20.515291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724694c4-3dbe-478a-8427-6d57687ff696 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:20.515582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724694c4-3dbe-478a-8427-6d57687ff696 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:20.515863 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724694c4-3dbe-478a-8427-6d57687ff696 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:20.516521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724694c4-3dbe-478a-8427-6d57687ff696 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:20.517881 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724694c4-3dbe-478a-8427-6d57687ff696 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:20.530370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-724694c4-3dbe-478a-8427-6d57687ff696 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 22:24:20.530370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:20.530370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:24:20.530370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:24:20.530370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:24:20.530370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:24:20.530370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:24:20.530370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:20.530370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 22:24:20.530370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:20.530370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:24:20.530370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:20.530370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:20.530370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:20.530370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:20.530370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:20.530370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:20.531583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:20.531583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:20.531583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:20.531583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:20.531583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:20.531583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:20.531583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:24:20.531583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:24:20.531583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:24:20.531583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:24:20.531583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:20.531583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:20.531583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:20.531583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:20.531583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:20.531583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:20.532500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:20.532500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:20.532500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:20.532500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:20.532500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:20.532500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:20.532500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:20.532500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:20.532500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:20.532500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:20.532500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:24:20.532500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:20.532500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 22:24:20.532500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:20.533803 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 494/1010] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:24:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:20.538062 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d7750-c72d-4450-a165-d2b2fb05e72c None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:20.548317 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:20.639114 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d7750-c72d-4450-a165-d2b2fb05e72c None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:20.675585 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa84ef-4f07-4a3a-9ced-29321da1fce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:20.676265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa84ef-4f07-4a3a-9ced-29321da1fce6 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:20.676553 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa84ef-4f07-4a3a-9ced-29321da1fce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:20.676835 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa84ef-4f07-4a3a-9ced-29321da1fce6 None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:20.677644 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa84ef-4f07-4a3a-9ced-29321da1fce6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:20.677960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa84ef-4f07-4a3a-9ced-29321da1fce6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:20.679365 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa84ef-4f07-4a3a-9ced-29321da1fce6 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:20.688732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 517/1011] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 22:24:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:20.689963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724694c4-3dbe-478a-8427-6d57687ff696 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:20.794146 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724694c4-3dbe-478a-8427-6d57687ff696 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:20.798606 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4452d90-1af1-4bdc-9b77-7bd68523d293 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:20.798897 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4452d90-1af1-4bdc-9b77-7bd68523d293 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:20.799206 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4452d90-1af1-4bdc-9b77-7bd68523d293 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:20.800719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 518/1012] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:24:20.805776 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de6da9-6497-4053-86f2-7af09a5e4713 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:20.806067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de6da9-6497-4053-86f2-7af09a5e4713 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:20.806354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4de6da9-6497-4053-86f2-7af09a5e4713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:20.823248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-c4de6da9-6497-4053-86f2-7af09a5e4713 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:20.824431 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-c4de6da9-6497-4053-86f2-7af09a5e4713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:20.830404 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28228cbf-87fe-4338-b7a8-75ca1fc5368b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:20.831100 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28228cbf-87fe-4338-b7a8-75ca1fc5368b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:20.831453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28228cbf-87fe-4338-b7a8-75ca1fc5368b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:20.831730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28228cbf-87fe-4338-b7a8-75ca1fc5368b None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:20.832421 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28228cbf-87fe-4338-b7a8-75ca1fc5368b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:20.832797 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28228cbf-87fe-4338-b7a8-75ca1fc5368b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:20.835169 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28228cbf-87fe-4338-b7a8-75ca1fc5368b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:20.840530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 495/1013] 10.222.0.43 () {56 vars in 1112 bytes} [Thu Jun 23 22:24:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:20.845728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28228cbf-87fe-4338-b7a8-75ca1fc5368b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:20.907381 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 519/1014] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:20.913040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:20.924579 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de6da9-6497-4053-86f2-7af09a5e4713 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:20.926706 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:20.960123 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28228cbf-87fe-4338-b7a8-75ca1fc5368b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:20.993145 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd03991a-b406-49e8-be70-c78d167fb9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:20.993801 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03991a-b406-49e8-be70-c78d167fb9f1 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:20.994085 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03991a-b406-49e8-be70-c78d167fb9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:20.994368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd03991a-b406-49e8-be70-c78d167fb9f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:20.995130 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03991a-b406-49e8-be70-c78d167fb9f1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:20.996479 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd03991a-b406-49e8-be70-c78d167fb9f1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:21.015031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 496/1015] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Jun 23 22:24:20 2022] POST /identity/v3/endpoints => generated 334 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:24:21.016503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390014f-5a93-4166-a80e-7529736c0ca5 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:21.016788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390014f-5a93-4166-a80e-7529736c0ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:21.017272 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390014f-5a93-4166-a80e-7529736c0ca5 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:21.018966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 497/1016] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:21.026610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39852c-d9b9-494c-9217-f2fa8fffca7f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:21.027027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39852c-d9b9-494c-9217-f2fa8fffca7f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:21.027313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39852c-d9b9-494c-9217-f2fa8fffca7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:21.031714 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4de6da9-6497-4053-86f2-7af09a5e4713 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:21.047628 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5e39852c-d9b9-494c-9217-f2fa8fffca7f None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:21.048821 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5e39852c-d9b9-494c-9217-f2fa8fffca7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:21.080347 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec3089-256a-42f2-acbe-7686ddf19e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:21.081036 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec3089-256a-42f2-acbe-7686ddf19e6c None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:21.081324 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec3089-256a-42f2-acbe-7686ddf19e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:21.081609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec3089-256a-42f2-acbe-7686ddf19e6c None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:21.082354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec3089-256a-42f2-acbe-7686ddf19e6c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:21.084477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec3089-256a-42f2-acbe-7686ddf19e6c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:21.101258 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 520/1017] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 22:24:20 2022] POST /identity/v3/services => generated 242 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:24:21.150746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 498/1018] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:21.175864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec3089-256a-42f2-acbe-7686ddf19e6c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:21.250208 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:21.296475 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec3089-256a-42f2-acbe-7686ddf19e6c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:21.330640 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb0c59-b333-425a-b434-514b80b50de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:21.331392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb0c59-b333-425a-b434-514b80b50de7 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:21.331679 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb0c59-b333-425a-b434-514b80b50de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:21.331957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb0c59-b333-425a-b434-514b80b50de7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:21.333124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb0c59-b333-425a-b434-514b80b50de7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:21.335229 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb0c59-b333-425a-b434-514b80b50de7 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:21.337155 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 521/1019] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:21.342656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39852c-d9b9-494c-9217-f2fa8fffca7f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:21.384886 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:21.445353 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39852c-d9b9-494c-9217-f2fa8fffca7f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:21.480829 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af622791-0b1d-4269-ab90-b4d57978ec02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:21.481497 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af622791-0b1d-4269-ab90-b4d57978ec02 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:21.481783 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af622791-0b1d-4269-ab90-b4d57978ec02 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:21.482060 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af622791-0b1d-4269-ab90-b4d57978ec02 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:21.482772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af622791-0b1d-4269-ab90-b4d57978ec02 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:21.484222 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af622791-0b1d-4269-ab90-b4d57978ec02 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:21.513034 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 499/1020] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 22:24:21 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:24:23.186178 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd908006-c7fb-4cd9-8909-59ae29a01674 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:23.186692 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd908006-c7fb-4cd9-8909-59ae29a01674 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:23.186770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd908006-c7fb-4cd9-8909-59ae29a01674 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:23.186877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ad049-013b-4b02-8823-feb158c0ca90 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:23.187241 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ad049-013b-4b02-8823-feb158c0ca90 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:23.187489 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ad049-013b-4b02-8823-feb158c0ca90 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:23.188364 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 522/1021] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:23.188886 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 500/1022] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:23.193680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86e93a-f8b4-455a-9f56-bf4db2e66103 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:23.193972 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86e93a-f8b4-455a-9f56-bf4db2e66103 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:23.194254 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86e93a-f8b4-455a-9f56-bf4db2e66103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:23.194565 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e25ea8-b448-4e36-b842-e3370db17cba None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:23.194864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e25ea8-b448-4e36-b842-e3370db17cba None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:23.195181 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e25ea8-b448-4e36-b842-e3370db17cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:23.212178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-25e25ea8-b448-4e36-b842-e3370db17cba None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:23.212352 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-cc86e93a-f8b4-455a-9f56-bf4db2e66103 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:23.213345 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-25e25ea8-b448-4e36-b842-e3370db17cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:23.213472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-cc86e93a-f8b4-455a-9f56-bf4db2e66103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:23.281439 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 523/1023] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:23.287030 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 501/1024] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:23.304472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86e93a-f8b4-455a-9f56-bf4db2e66103 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:23.310254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e25ea8-b448-4e36-b842-e3370db17cba None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:23.336699 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:23.367245 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:23.414700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86e93a-f8b4-455a-9f56-bf4db2e66103 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:23.421017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e25ea8-b448-4e36-b842-e3370db17cba None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:23.450509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721574e-bbaa-4ef5-ace5-46066b24df77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:23.451346 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721574e-bbaa-4ef5-ace5-46066b24df77 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:23.451645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721574e-bbaa-4ef5-ace5-46066b24df77 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:23.451933 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721574e-bbaa-4ef5-ace5-46066b24df77 None admin] PATH_INFO: `/v3/services/image` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:23.452598 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721574e-bbaa-4ef5-ace5-46066b24df77 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:23.453978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721574e-bbaa-4ef5-ace5-46066b24df77 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:23.457425 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7059db-612d-4ae5-8930-74f111ccb337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:23.458231 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7059db-612d-4ae5-8930-74f111ccb337 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:23.459057 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7059db-612d-4ae5-8930-74f111ccb337 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:23.459388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7059db-612d-4ae5-8930-74f111ccb337 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:23.466559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-7721574e-bbaa-4ef5-ace5-46066b24df77 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 22:24:23.466559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:23.466559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:24:23.466559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:24:23.466559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:24:23.466559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:24:23.466559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:24:23.466559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:23.466559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 22:24:23.466559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:23.466559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:24:23.466559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:23.466559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:23.466559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:23.466559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:23.466559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:23.466559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:23.467594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:23.467594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:23.467594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:23.467594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:23.467594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:23.467594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:23.467594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:24:23.467594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:24:23.467594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:24:23.467594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:24:23.467594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:23.467594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:23.467594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:23.467594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:23.467594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:23.467594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:23.468453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:23.468453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:23.468453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:23.468453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:23.468453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:23.468453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:23.468453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:23.468453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:23.468453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:23.468453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:23.468453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:24:23.468453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:23.468453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 22:24:23.468453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:23.468453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 524/1025] 10.222.0.43 () {56 vars in 1105 bytes} [Thu Jun 23 22:24:23 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:23.473081 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7059db-612d-4ae5-8930-74f111ccb337 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:24:23.475830 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721574e-bbaa-4ef5-ace5-46066b24df77 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:23.482080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-7a7059db-612d-4ae5-8930-74f111ccb337 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:24:23.482080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:23.482080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:23.482080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:23.482080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:23.482080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:23.482080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:23.482080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:23.482080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:23.482080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:23.482080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:23.482080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:23.482080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:24:23.482080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:24:23.482080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:24:23.482080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:24:23.483078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:24:23.483078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:24:23.483078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:24:23.483078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:24:23.483078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:23.483078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:23.483078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:23.483078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:23.483078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:23.483078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:23.483078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:23.483078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:23.483078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:23.483078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:23.483078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:23.483078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:23.483958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:23.483958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:23.483958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:23.483958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:23.483958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:24:23.483958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:24:23.483958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:24:23.483958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:24:23.483958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:24:23.483958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:24:23.483958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:24:23.483958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:23.487282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 502/1026] 10.222.0.43 () {56 vars in 1120 bytes} [Thu Jun 23 22:24:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:23.489153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7059db-612d-4ae5-8930-74f111ccb337 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:23.578315 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:23.589996 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721574e-bbaa-4ef5-ace5-46066b24df77 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:23.598868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7059db-612d-4ae5-8930-74f111ccb337 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:23.610500 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:23.625402 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1e124-73a0-4a17-8f4e-8ed7df1c7202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:23.626170 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1e124-73a0-4a17-8f4e-8ed7df1c7202 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:23.626460 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1e124-73a0-4a17-8f4e-8ed7df1c7202 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:23.626750 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1e124-73a0-4a17-8f4e-8ed7df1c7202 None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:23.627496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1e124-73a0-4a17-8f4e-8ed7df1c7202 None admin] RBAC: Adding query filter params (name=image) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:23.627809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1e124-73a0-4a17-8f4e-8ed7df1c7202 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:23.629260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1e124-73a0-4a17-8f4e-8ed7df1c7202 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:23.638948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 525/1027] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:23.640129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1e124-73a0-4a17-8f4e-8ed7df1c7202 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:23.640653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919fe5ac-68c2-4787-b41f-b4995a5dccb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:23.641332 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fe5ac-68c2-4787-b41f-b4995a5dccb4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:23.641624 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fe5ac-68c2-4787-b41f-b4995a5dccb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:23.641910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fe5ac-68c2-4787-b41f-b4995a5dccb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:23.642670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919fe5ac-68c2-4787-b41f-b4995a5dccb4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:23.643021 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919fe5ac-68c2-4787-b41f-b4995a5dccb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:23.644406 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919fe5ac-68c2-4787-b41f-b4995a5dccb4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:23.658000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 503/1028] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Jun 23 22:24:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:23.658000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d394d-f83d-48a0-9c6a-6e5177f024b6 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:23.658000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d394d-f83d-48a0-9c6a-6e5177f024b6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:23.658000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d394d-f83d-48a0-9c6a-6e5177f024b6 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:23.658000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 504/1029] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:23.659466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d394d-f83d-48a0-9c6a-6e5177f024b6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:23.745110 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1e124-73a0-4a17-8f4e-8ed7df1c7202 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:23.760458 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d394d-f83d-48a0-9c6a-6e5177f024b6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:23.778364 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21a98c-0b05-4ca3-a6d4-fc698ca0d683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:23.779072 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21a98c-0b05-4ca3-a6d4-fc698ca0d683 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:23.779355 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21a98c-0b05-4ca3-a6d4-fc698ca0d683 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:23.779633 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21a98c-0b05-4ca3-a6d4-fc698ca0d683 None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:23.780351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21a98c-0b05-4ca3-a6d4-fc698ca0d683 None admin] RBAC: Adding query filter params (type=image) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:23.780659 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21a98c-0b05-4ca3-a6d4-fc698ca0d683 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:23.782022 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21a98c-0b05-4ca3-a6d4-fc698ca0d683 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:23.786689 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 526/1030] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:23.787864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b88d9-ef31-4378-8d20-0745c27239b3 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:23.788152 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b88d9-ef31-4378-8d20-0745c27239b3 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:23.788433 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b88d9-ef31-4378-8d20-0745c27239b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:23.800740 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf488e-8666-4cc8-9378-8dc61796d87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:23.801540 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf488e-8666-4cc8-9378-8dc61796d87c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:23.801829 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf488e-8666-4cc8-9378-8dc61796d87c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:23.802147 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf488e-8666-4cc8-9378-8dc61796d87c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:23.804040 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-5c6b88d9-ef31-4378-8d20-0745c27239b3 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:23.805216 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-5c6b88d9-ef31-4378-8d20-0745c27239b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:23.812579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf488e-8666-4cc8-9378-8dc61796d87c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:23.814274 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf488e-8666-4cc8-9378-8dc61796d87c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:23.816629 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:23.835099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-66bf488e-8666-4cc8-9378-8dc61796d87c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:23.835099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:23.835099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:23.835099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:23.835099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:23.835099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:23.835099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:23.835099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:23.835099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:23.835099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:23.835099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:23.835099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:23.835099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:24:23.835099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:24:23.835099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:24:23.836902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:24:23.836902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:23.836902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:23.836902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:23.836902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:23.836902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:23.836902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:23.836902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:23.836902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:23.836902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:23.836902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:23.836902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:23.836902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:23.836902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:23.836902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:23.836902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:23.836902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:23.837814 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:24:23.837814 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:24:23.837814 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:23.837814 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:23.837814 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 505/1031] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:23.837814 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf488e-8666-4cc8-9378-8dc61796d87c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:23.856880 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:23.884044 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 527/1032] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:23.885202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b88d9-ef31-4378-8d20-0745c27239b3 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:23.947179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf488e-8666-4cc8-9378-8dc61796d87c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:23.980638 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deae20a-3634-4c81-80ea-40ec757e0682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:23.981304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deae20a-3634-4c81-80ea-40ec757e0682 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:23.981586 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deae20a-3634-4c81-80ea-40ec757e0682 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:23.981872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deae20a-3634-4c81-80ea-40ec757e0682 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:23.982653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deae20a-3634-4c81-80ea-40ec757e0682 None admin] RBAC: Adding query filter params (interface=public, service_id=a5c433a07611448ab87221a9b3fc5f66, region_id=RegionOne) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:23.983000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deae20a-3634-4c81-80ea-40ec757e0682 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:23.984367 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deae20a-3634-4c81-80ea-40ec757e0682 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:23.987324 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b88d9-ef31-4378-8d20-0745c27239b3 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:23.989773 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 506/1033] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Jun 23 22:24:23 2022] GET /identity/v3/endpoints?service_id=a5c433a07611448ab87221a9b3fc5f66&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:23.990684 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deae20a-3634-4c81-80ea-40ec757e0682 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:24.024408 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551dffb-d403-422c-8925-a19f6915260a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:24.025012 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551dffb-d403-422c-8925-a19f6915260a None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:24.025299 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551dffb-d403-422c-8925-a19f6915260a None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:24.025641 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551dffb-d403-422c-8925-a19f6915260a None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:24.026366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551dffb-d403-422c-8925-a19f6915260a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:24.026682 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551dffb-d403-422c-8925-a19f6915260a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:24.028103 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551dffb-d403-422c-8925-a19f6915260a None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:24.039771 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 528/1034] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:24.040592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551dffb-d403-422c-8925-a19f6915260a None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:24.054644 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:24.080132 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:24.096397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deae20a-3634-4c81-80ea-40ec757e0682 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:24.132809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d16f5-959e-4457-a034-3ce2c7912242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:24.134028 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d16f5-959e-4457-a034-3ce2c7912242 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:24.134315 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d16f5-959e-4457-a034-3ce2c7912242 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:24.134597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d16f5-959e-4457-a034-3ce2c7912242 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:24.135924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d16f5-959e-4457-a034-3ce2c7912242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:24.137301 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5d16f5-959e-4457-a034-3ce2c7912242 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:24.138953 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 507/1035] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:24.139711 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d16f5-959e-4457-a034-3ce2c7912242 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:24.147589 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551dffb-d403-422c-8925-a19f6915260a None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:24.183267 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523281c-80d5-45c8-822a-bca7dbf6008d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:24.183947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523281c-80d5-45c8-822a-bca7dbf6008d None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:24.184227 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523281c-80d5-45c8-822a-bca7dbf6008d None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:24.184511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523281c-80d5-45c8-822a-bca7dbf6008d None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:24.185193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523281c-80d5-45c8-822a-bca7dbf6008d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:24.186562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523281c-80d5-45c8-822a-bca7dbf6008d None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:24.193350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 529/1036] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Jun 23 22:24:24 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:24:24.194062 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523281c-80d5-45c8-822a-bca7dbf6008d None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:24.245418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d16f5-959e-4457-a034-3ce2c7912242 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:24.278330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:24.288497 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a2d00-61a2-4fc7-a760-76a1714a09c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:24.289420 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a2d00-61a2-4fc7-a760-76a1714a09c2 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:24.289779 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a2d00-61a2-4fc7-a760-76a1714a09c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:24.290075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a2d00-61a2-4fc7-a760-76a1714a09c2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:24.298051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a2d00-61a2-4fc7-a760-76a1714a09c2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:24.299561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a2d00-61a2-4fc7-a760-76a1714a09c2 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:24.315109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3d2a2d00-61a2-4fc7-a760-76a1714a09c2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:24:24.315109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:24.315109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:24.315109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:24.315109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:24.315109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:24.315109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:24.315109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:24.315109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:24.315109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:24.315109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:24.315109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:24.315109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:24:24.315109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:24:24.315109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:24:24.315109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:24:24.316173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:24.316173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:24.316173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:24:24.316173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:24:24.316173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:24:24.316173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:24:24.316173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:24.316173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:24.316173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:24.316173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:24.316173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:24.316173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:24.316173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:24.316173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:24.316173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:24.316173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:24.316173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:24.316173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:24.317137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:24.317137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:24.317137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:24:24.317137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:24:24.317137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:24:24.317137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:24:24.317137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:24:24.317137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:24:24.317137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:24:24.317137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:24.317137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 508/1037] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Jun 23 22:24:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:24.323633 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a2d00-61a2-4fc7-a760-76a1714a09c2 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:24.325548 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523281c-80d5-45c8-822a-bca7dbf6008d None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:24.328294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:24.370093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ad111-c525-4ca9-b7cd-fb3b6b5a5afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:24.370788 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ad111-c525-4ca9-b7cd-fb3b6b5a5afc None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:24.371113 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ad111-c525-4ca9-b7cd-fb3b6b5a5afc None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:24.371407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ad111-c525-4ca9-b7cd-fb3b6b5a5afc None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:24.372136 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ad111-c525-4ca9-b7cd-fb3b6b5a5afc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:24.373583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ad111-c525-4ca9-b7cd-fb3b6b5a5afc None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:24.393826 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 530/1038] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 22:24:24 2022] POST /identity/v3/users => generated 285 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:24:24.437399 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a2d00-61a2-4fc7-a760-76a1714a09c2 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:24.476772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eac7e2-7631-49f4-b0d1-9d56efb8557f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:24.477471 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eac7e2-7631-49f4-b0d1-9d56efb8557f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:24.477760 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eac7e2-7631-49f4-b0d1-9d56efb8557f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:24.478050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eac7e2-7631-49f4-b0d1-9d56efb8557f None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:24.478914 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eac7e2-7631-49f4-b0d1-9d56efb8557f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:24.479260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eac7e2-7631-49f4-b0d1-9d56efb8557f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:24.480743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eac7e2-7631-49f4-b0d1-9d56efb8557f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:24.505113 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 509/1039] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Jun 23 22:24:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:24.507130 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ad111-c525-4ca9-b7cd-fb3b6b5a5afc None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:24.616153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ad111-c525-4ca9-b7cd-fb3b6b5a5afc None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:24.653666 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e43c9-8e8d-462f-9ca3-236997a6acd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:24.654335 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e43c9-8e8d-462f-9ca3-236997a6acd0 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:24.654623 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e43c9-8e8d-462f-9ca3-236997a6acd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:24.654905 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e43c9-8e8d-462f-9ca3-236997a6acd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:24.686315 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:24.731756 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e43c9-8e8d-462f-9ca3-236997a6acd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:24.733212 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e43c9-8e8d-462f-9ca3-236997a6acd0 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:24.838005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 531/1040] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:24:24 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:24:24.843416 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eac7e2-7631-49f4-b0d1-9d56efb8557f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:24.894275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:24.951411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eac7e2-7631-49f4-b0d1-9d56efb8557f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:24.984756 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cb5b86-2b83-45cc-a5ad-7e79a35122c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:24.985551 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb5b86-2b83-45cc-a5ad-7e79a35122c1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:24.985837 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb5b86-2b83-45cc-a5ad-7e79a35122c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:24.986115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb5b86-2b83-45cc-a5ad-7e79a35122c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:24.996510 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cb5b86-2b83-45cc-a5ad-7e79a35122c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:24.997938 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cb5b86-2b83-45cc-a5ad-7e79a35122c1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:25.015370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f6cb5b86-2b83-45cc-a5ad-7e79a35122c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:25.015370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:25.015370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:25.015370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:25.015370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:25.015370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:25.015370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:25.015370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:25.015370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:25.015370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:25.015370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:25.015370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:25.015370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:24:25.015370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:24:25.015370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:24:25.016278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:24:25.016278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:25.016278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:25.016278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:25.016278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:25.016278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:25.016278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:25.016278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:25.016278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:25.016278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:25.016278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:25.016278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:25.016278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:25.016278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:25.016278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:25.016278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:25.016278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:25.017185 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:24:25.017185 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:24:25.017185 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:25.017185 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:25.019242 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 510/1041] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:25.022150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e43c9-8e8d-462f-9ca3-236997a6acd0 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:25.081670 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:25.134635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e43c9-8e8d-462f-9ca3-236997a6acd0 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:25.170208 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090c7b9-eef0-45c5-9f4a-db761f8276b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:25.170801 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090c7b9-eef0-45c5-9f4a-db761f8276b2 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:25.171188 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090c7b9-eef0-45c5-9f4a-db761f8276b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:25.171476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2090c7b9-eef0-45c5-9f4a-db761f8276b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:25.172205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090c7b9-eef0-45c5-9f4a-db761f8276b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:25.172519 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090c7b9-eef0-45c5-9f4a-db761f8276b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:25.173895 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2090c7b9-eef0-45c5-9f4a-db761f8276b2 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:25.192709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 532/1042] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:25.194269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cb5b86-2b83-45cc-a5ad-7e79a35122c1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:25.293699 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:25.299031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cb5b86-2b83-45cc-a5ad-7e79a35122c1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:25.336160 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77718d6f-e490-44bb-bb84-a43ebae28087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:25.336965 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77718d6f-e490-44bb-bb84-a43ebae28087 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:25.337257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77718d6f-e490-44bb-bb84-a43ebae28087 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:25.337539 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77718d6f-e490-44bb-bb84-a43ebae28087 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:25.344179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77718d6f-e490-44bb-bb84-a43ebae28087 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:25.345626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77718d6f-e490-44bb-bb84-a43ebae28087 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:25.360156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-77718d6f-e490-44bb-bb84-a43ebae28087 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:24:25.360156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:25.360156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:25.360156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:25.360156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:25.360156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:25.360156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:25.360156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:25.360156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:25.360156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:25.360156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:25.360156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:25.360156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:24:25.360156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:24:25.360156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:24:25.361070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:24:25.361070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:25.361070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:25.361070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:25.361070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:25.361070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:25.361070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:25.361070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:25.361070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:25.361070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:25.361070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:25.361070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:25.361070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:25.361070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:25.361070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:25.361070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:25.361070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:25.367626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:24:25.367626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:24:25.367626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:24:25.367626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:24:25.367626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:24:25.367626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:24:25.367626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:24:25.367626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:25.367626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 511/1043] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:24:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:25.368116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090c7b9-eef0-45c5-9f4a-db761f8276b2 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:25.469241 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2090c7b9-eef0-45c5-9f4a-db761f8276b2 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:25.485945 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:25.504248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04225b27-ef10-4c86-9742-9f7ad337d3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:25.504920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04225b27-ef10-4c86-9742-9f7ad337d3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:25.505731 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04225b27-ef10-4c86-9742-9f7ad337d3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:25.506384 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04225b27-ef10-4c86-9742-9f7ad337d3d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:25.507009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04225b27-ef10-4c86-9742-9f7ad337d3d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:25.507331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04225b27-ef10-4c86-9742-9f7ad337d3d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:25.508774 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04225b27-ef10-4c86-9742-9f7ad337d3d3 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:25.526944 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 533/1044] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:24:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:25.528185 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77718d6f-e490-44bb-bb84-a43ebae28087 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:25.639805 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77718d6f-e490-44bb-bb84-a43ebae28087 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:25.672958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077c9c0-418e-4411-a259-1e3339221872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:25.673615 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077c9c0-418e-4411-a259-1e3339221872 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:25.673900 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077c9c0-418e-4411-a259-1e3339221872 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:25.674181 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077c9c0-418e-4411-a259-1e3339221872 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:25.675019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077c9c0-418e-4411-a259-1e3339221872 None admin] RBAC: Adding query filter params (role.id=26dbcfd7d83742b1b1cd311915a9c0a8, scope.project.id=56da0975019c44409b53be473131143a, user.id=35d686923ef541819de3c77847d38511) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:25.675338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077c9c0-418e-4411-a259-1e3339221872 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:25.676749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077c9c0-418e-4411-a259-1e3339221872 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:25.686999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 512/1045] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:24:25 2022] GET /identity/v3/role_assignments?user.id=35d686923ef541819de3c77847d38511&scope.project.id=56da0975019c44409b53be473131143a&role.id=26dbcfd7d83742b1b1cd311915a9c0a8 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:26.252872 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256763d-8e6f-4d67-9803-09bb0a8fb92e None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:26.253383 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256763d-8e6f-4d67-9803-09bb0a8fb92e None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:26.253462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256763d-8e6f-4d67-9803-09bb0a8fb92e None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:26.254808 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 534/1046] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:26.260077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbede29-90ba-4215-bbad-ad9995b5a8bc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:26.260381 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbede29-90ba-4215-bbad-ad9995b5a8bc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:26.260668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbede29-90ba-4215-bbad-ad9995b5a8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:26.277716 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bbbede29-90ba-4215-bbad-ad9995b5a8bc None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:26.278905 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bbbede29-90ba-4215-bbad-ad9995b5a8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:26.296482 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:26.348627 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 513/1047] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:26.371486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256763d-8e6f-4d67-9803-09bb0a8fb92e None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:26.473563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256763d-8e6f-4d67-9803-09bb0a8fb92e None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:26.507619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581cd0fd-d56d-4c87-8a0a-6e7030bdcc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:26.508410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cd0fd-d56d-4c87-8a0a-6e7030bdcc29 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:26.508691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cd0fd-d56d-4c87-8a0a-6e7030bdcc29 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:26.508963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cd0fd-d56d-4c87-8a0a-6e7030bdcc29 None admin] PATH_INFO: `/v3/services/image` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:26.509605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cd0fd-d56d-4c87-8a0a-6e7030bdcc29 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:26.511000 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cd0fd-d56d-4c87-8a0a-6e7030bdcc29 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:26.522447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-581cd0fd-d56d-4c87-8a0a-6e7030bdcc29 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 22:24:26.522447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:26.522447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:24:26.522447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:24:26.522447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:24:26.522447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:24:26.522447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:24:26.522447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:26.522447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 22:24:26.522447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:26.522447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:24:26.522447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:26.522447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:26.522447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:26.522447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:26.522447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:26.522447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:26.523766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:26.523766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:26.523766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:26.523766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:26.523766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:26.523766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:26.523766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:24:26.523766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:24:26.523766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:24:26.523766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:24:26.523766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:26.523766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:26.523766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:26.523766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:26.523766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:26.523766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:26.524669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:26.524669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:26.524669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:26.524669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:26.524669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:26.524669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:26.524669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:26.524669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:26.524669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:26.524669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:26.524669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:24:26.524669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:24:26.524669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 22:24:26.524669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:26.527233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 535/1048] 10.222.0.43 () {56 vars in 1105 bytes} [Thu Jun 23 22:24:26 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:26.529816 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbede29-90ba-4215-bbad-ad9995b5a8bc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:26.545206 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97cf22b-146d-4a33-ad56-e647e9d3b2c4 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:26.545499 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97cf22b-146d-4a33-ad56-e647e9d3b2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:26.545789 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97cf22b-146d-4a33-ad56-e647e9d3b2c4 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:26.547205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 536/1049] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:24:26.554039 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c847d-8b01-47f2-9d97-8a4df0a78484 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:26.554337 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c847d-8b01-47f2-9d97-8a4df0a78484 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:26.554628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c847d-8b01-47f2-9d97-8a4df0a78484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:26.573283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-d97c847d-8b01-47f2-9d97-8a4df0a78484 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:26.574465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d97c847d-8b01-47f2-9d97-8a4df0a78484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:26.612486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:26.639459 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbede29-90ba-4215-bbad-ad9995b5a8bc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:26.651279 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 537/1050] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:26 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:26.672999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c847d-8b01-47f2-9d97-8a4df0a78484 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:26.678063 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c79c6-64b4-437e-96c2-8bcb9e430fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:26.678764 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c79c6-64b4-437e-96c2-8bcb9e430fba None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:26.679105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c79c6-64b4-437e-96c2-8bcb9e430fba None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:26.679395 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c79c6-64b4-437e-96c2-8bcb9e430fba None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:26.680112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c79c6-64b4-437e-96c2-8bcb9e430fba None admin] RBAC: Adding query filter params (name=image) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:26.680429 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c79c6-64b4-437e-96c2-8bcb9e430fba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:26.681850 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c79c6-64b4-437e-96c2-8bcb9e430fba None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:26.687172 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 514/1051] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:26.692871 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c79c6-64b4-437e-96c2-8bcb9e430fba None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:26.710437 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:26.793264 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c847d-8b01-47f2-9d97-8a4df0a78484 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:26.796456 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c79c6-64b4-437e-96c2-8bcb9e430fba None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:26.829480 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc087a-e7cf-4a7f-bbf4-809648ff52c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:26.830278 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc087a-e7cf-4a7f-bbf4-809648ff52c4 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:26.830568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc087a-e7cf-4a7f-bbf4-809648ff52c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:26.830848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc087a-e7cf-4a7f-bbf4-809648ff52c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:26.832901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a33980-2a24-4267-9482-bf3cf5525062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:26.834329 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a33980-2a24-4267-9482-bf3cf5525062 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:26.834641 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a33980-2a24-4267-9482-bf3cf5525062 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:26.834985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a33980-2a24-4267-9482-bf3cf5525062 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:26.835893 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a33980-2a24-4267-9482-bf3cf5525062 None admin] RBAC: Adding query filter params (type=image) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:26.836211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a33980-2a24-4267-9482-bf3cf5525062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:26.837639 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a33980-2a24-4267-9482-bf3cf5525062 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:26.842206 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc087a-e7cf-4a7f-bbf4-809648ff52c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:24:26.847406 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 515/1052] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:26.848197 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a33980-2a24-4267-9482-bf3cf5525062 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:26.850205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-b2bc087a-e7cf-4a7f-bbf4-809648ff52c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:24:26.850205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:26.850205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:26.850205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:26.850205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:26.850205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:26.850205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:26.850205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:26.850205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:26.850205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:26.850205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:26.850205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:26.850205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:24:26.850205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:24:26.850205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:24:26.850205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:24:26.851182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:24:26.851182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:24:26.851182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:24:26.851182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:24:26.851182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:26.851182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:26.851182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:26.851182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:26.851182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:26.851182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:26.851182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:26.851182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:26.851182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:26.851182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:26.851182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:26.851182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:26.852031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:26.852031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:26.852031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:26.852031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:26.852031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:24:26.852031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:24:26.852031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:24:26.852031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:24:26.852031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:24:26.852031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:24:26.852031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:24:26.852031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:26.852031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 538/1053] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:24:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:26.857310 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc087a-e7cf-4a7f-bbf4-809648ff52c4 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:26.889543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:26.935007 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:26.961213 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a33980-2a24-4267-9482-bf3cf5525062 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:26.967764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc087a-e7cf-4a7f-bbf4-809648ff52c4 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:27.010606 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3afced0-8e5e-4352-9228-ac8f027f4f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:27.011294 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afced0-8e5e-4352-9228-ac8f027f4f49 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:27.011582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afced0-8e5e-4352-9228-ac8f027f4f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:27.011863 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afced0-8e5e-4352-9228-ac8f027f4f49 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:27.012596 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3afced0-8e5e-4352-9228-ac8f027f4f49 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:27.012760 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bbb42-e497-4c7f-aa0e-5141fe0a0356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:27.013464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bbb42-e497-4c7f-aa0e-5141fe0a0356 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:27.013757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bbb42-e497-4c7f-aa0e-5141fe0a0356 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:27.013951 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3afced0-8e5e-4352-9228-ac8f027f4f49 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:27.014047 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4bbb42-e497-4c7f-aa0e-5141fe0a0356 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:27.014836 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4bbb42-e497-4c7f-aa0e-5141fe0a0356 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:27.015118 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4bbb42-e497-4c7f-aa0e-5141fe0a0356 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:27.016476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4bbb42-e497-4c7f-aa0e-5141fe0a0356 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:27.030874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 539/1054] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:24:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:27.031939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bbb42-e497-4c7f-aa0e-5141fe0a0356 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:27.047035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 516/1055] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Jun 23 22:24:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:24:27.139723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:27.148534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4bbb42-e497-4c7f-aa0e-5141fe0a0356 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:27.193914 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bcab40-4bf4-4dec-ac00-c77da37d72bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:27.194810 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcab40-4bf4-4dec-ac00-c77da37d72bf None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:27.195133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcab40-4bf4-4dec-ac00-c77da37d72bf None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:27.195405 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcab40-4bf4-4dec-ac00-c77da37d72bf None admin] PATH_INFO: `/v3/users/399341e6554244a297ccce389c17bc09` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:27.209769 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bcab40-4bf4-4dec-ac00-c77da37d72bf None admin] RBAC: Authorizing `identity:get_user(user_id=399341e6554244a297ccce389c17bc09)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:27.211277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bcab40-4bf4-4dec-ac00-c77da37d72bf None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:27.212961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 540/1056] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:24:27 2022] GET /identity/v3/users/399341e6554244a297ccce389c17bc09 => generated 285 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:27.218703 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3afced0-8e5e-4352-9228-ac8f027f4f49 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:27.327516 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3afced0-8e5e-4352-9228-ac8f027f4f49 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:27.362041 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ca4a5-a992-4072-8868-670c133923cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:27.362699 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ca4a5-a992-4072-8868-670c133923cb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:27.363016 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ca4a5-a992-4072-8868-670c133923cb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:27.363305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ca4a5-a992-4072-8868-670c133923cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:27.367167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:27.435347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ca4a5-a992-4072-8868-670c133923cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:27.436793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ca4a5-a992-4072-8868-670c133923cb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:27.538746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 517/1057] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:24:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:24:27.544577 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bcab40-4bf4-4dec-ac00-c77da37d72bf None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:27.651962 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bcab40-4bf4-4dec-ac00-c77da37d72bf None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:27.686121 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8dada4-78d0-4f42-8a3d-bb0483af72fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:27.687017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8dada4-78d0-4f42-8a3d-bb0483af72fa None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:27.687307 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8dada4-78d0-4f42-8a3d-bb0483af72fa None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:27.687590 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8dada4-78d0-4f42-8a3d-bb0483af72fa None admin] PATH_INFO: `/v3/projects/c4a7b621f2b149eb9e972bb032e21bda` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:27.688781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8dada4-78d0-4f42-8a3d-bb0483af72fa None admin] RBAC: Authorizing `identity:get_project(project_id=c4a7b621f2b149eb9e972bb032e21bda)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:27.690223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8dada4-78d0-4f42-8a3d-bb0483af72fa None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:27.697388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 541/1058] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:24:27 2022] GET /identity/v3/projects/c4a7b621f2b149eb9e972bb032e21bda => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:27.698645 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ca4a5-a992-4072-8868-670c133923cb None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:27.716548 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84c4c3-3aa2-4080-8d17-1556fc5e675e None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:27.716882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84c4c3-3aa2-4080-8d17-1556fc5e675e None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:27.717114 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84c4c3-3aa2-4080-8d17-1556fc5e675e None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:27.718467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 542/1059] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:24:27.723408 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a12366-40a6-4f85-a67f-ef45b6c1d725 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:27.723692 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a12366-40a6-4f85-a67f-ef45b6c1d725 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:27.724012 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a12366-40a6-4f85-a67f-ef45b6c1d725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:27.737567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:27.741089 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-a1a12366-40a6-4f85-a67f-ef45b6c1d725 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:27.742300 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-a1a12366-40a6-4f85-a67f-ef45b6c1d725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:27.753571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:27.811176 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ca4a5-a992-4072-8868-670c133923cb None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:27.814775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 543/1060] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:27.838743 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a12366-40a6-4f85-a67f-ef45b6c1d725 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:27.850855 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201020e-8f2f-474c-9850-c7b0fead5f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:27.851555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201020e-8f2f-474c-9850-c7b0fead5f4b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:27.851844 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201020e-8f2f-474c-9850-c7b0fead5f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:27.852128 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201020e-8f2f-474c-9850-c7b0fead5f4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:27.852997 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201020e-8f2f-474c-9850-c7b0fead5f4b None admin] RBAC: Adding query filter params (role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.project.id=c4a7b621f2b149eb9e972bb032e21bda, user.id=399341e6554244a297ccce389c17bc09) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:27.853315 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201020e-8f2f-474c-9850-c7b0fead5f4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:27.854824 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201020e-8f2f-474c-9850-c7b0fead5f4b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:27.870963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 518/1061] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:24:27 2022] GET /identity/v3/role_assignments?user.id=399341e6554244a297ccce389c17bc09&scope.project.id=c4a7b621f2b149eb9e972bb032e21bda&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:27.944886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a12366-40a6-4f85-a67f-ef45b6c1d725 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:27.980923 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31cb12-6b3d-4b2a-a5c4-d9ae6dd750fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:27.981758 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31cb12-6b3d-4b2a-a5c4-d9ae6dd750fc None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:27.982043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31cb12-6b3d-4b2a-a5c4-d9ae6dd750fc None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:27.982319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31cb12-6b3d-4b2a-a5c4-d9ae6dd750fc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:27.993235 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31cb12-6b3d-4b2a-a5c4-d9ae6dd750fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:24:28.001490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-dd31cb12-6b3d-4b2a-a5c4-d9ae6dd750fc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:24:28.001490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:28.001490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:28.001490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:28.001490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:28.001490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:28.001490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:28.001490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:28.001490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:28.001490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:28.001490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:28.001490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:28.001490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:24:28.001490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:24:28.001490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:24:28.001490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:24:28.002503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:24:28.002503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:24:28.002503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:24:28.002503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:24:28.002503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:28.002503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:28.002503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:28.002503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:28.002503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:28.002503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:28.002503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:28.002503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:28.002503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:28.002503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:28.002503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:28.002503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:28.003387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:28.003387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:28.003387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:28.003387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:28.003387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:24:28.003387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:24:28.003387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:24:28.003387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:24:28.003387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:24:28.003387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:24:28.003387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:24:28.003387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:28.003387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 544/1062] 10.222.0.43 () {56 vars in 1120 bytes} [Thu Jun 23 22:24:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:28.009237 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201020e-8f2f-474c-9850-c7b0fead5f4b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:28.114776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:28.118476 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201020e-8f2f-474c-9850-c7b0fead5f4b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:28.158645 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7824ce-845f-44f3-8bfd-53a1d8d5d4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:28.159361 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7824ce-845f-44f3-8bfd-53a1d8d5d4cd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:28.159692 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7824ce-845f-44f3-8bfd-53a1d8d5d4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:28.159974 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7824ce-845f-44f3-8bfd-53a1d8d5d4cd None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:28.160720 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7824ce-845f-44f3-8bfd-53a1d8d5d4cd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:28.161031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7824ce-845f-44f3-8bfd-53a1d8d5d4cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:28.163162 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7824ce-845f-44f3-8bfd-53a1d8d5d4cd None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:28.172135 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 519/1063] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Jun 23 22:24:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:28.178366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31cb12-6b3d-4b2a-a5c4-d9ae6dd750fc None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:28.188916 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:28.285738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31cb12-6b3d-4b2a-a5c4-d9ae6dd750fc None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:28.319989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630246e-6526-4422-9c9b-7d7ceb145794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:28.320686 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630246e-6526-4422-9c9b-7d7ceb145794 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:28.320968 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630246e-6526-4422-9c9b-7d7ceb145794 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:28.321244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630246e-6526-4422-9c9b-7d7ceb145794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:28.332009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630246e-6526-4422-9c9b-7d7ceb145794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:28.334193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630246e-6526-4422-9c9b-7d7ceb145794 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:28.351354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-4630246e-6526-4422-9c9b-7d7ceb145794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:28.351354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:28.351354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:28.351354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:28.351354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:28.351354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:28.351354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:28.351354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:28.351354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:28.351354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:28.351354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:28.351354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:28.351354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:24:28.351354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:24:28.351354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:24:28.352250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:24:28.352250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:28.352250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:28.352250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:28.352250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:28.352250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:28.352250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:28.352250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:28.352250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:28.352250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:28.352250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:28.352250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:28.352250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:28.352250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:28.352250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:28.352250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:28.352250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:28.353176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:24:28.353176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:24:28.353176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:28.353176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:28.353176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 545/1064] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:28.358542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7824ce-845f-44f3-8bfd-53a1d8d5d4cd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:28.459087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7824ce-845f-44f3-8bfd-53a1d8d5d4cd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:28.491857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37584bc4-9bbf-4008-af67-1ae422aac30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:28.492520 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37584bc4-9bbf-4008-af67-1ae422aac30a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:28.492805 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37584bc4-9bbf-4008-af67-1ae422aac30a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:28.493086 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37584bc4-9bbf-4008-af67-1ae422aac30a None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:28.493804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37584bc4-9bbf-4008-af67-1ae422aac30a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:28.494103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37584bc4-9bbf-4008-af67-1ae422aac30a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:28.495497 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37584bc4-9bbf-4008-af67-1ae422aac30a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:28.512341 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 520/1065] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:28.513836 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630246e-6526-4422-9c9b-7d7ceb145794 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:28.576215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:28.623867 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630246e-6526-4422-9c9b-7d7ceb145794 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:28.658092 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b9fbc-b761-43d3-9304-b229295cf64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:28.658892 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b9fbc-b761-43d3-9304-b229295cf64c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:28.659209 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b9fbc-b761-43d3-9304-b229295cf64c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:28.659492 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b9fbc-b761-43d3-9304-b229295cf64c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:28.667255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b9fbc-b761-43d3-9304-b229295cf64c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:28.668700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b9fbc-b761-43d3-9304-b229295cf64c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:28.683320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-c94b9fbc-b761-43d3-9304-b229295cf64c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:24:28.683320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:28.683320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:28.683320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:28.683320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:28.683320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:28.683320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:28.683320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:28.683320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:28.683320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:28.683320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:28.683320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:28.683320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:24:28.683320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:24:28.683320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:24:28.683320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:24:28.684292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:28.684292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:28.684292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:24:28.684292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:24:28.684292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:24:28.684292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:24:28.684292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:28.684292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:28.684292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:28.684292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:28.684292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:28.684292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:28.684292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:28.684292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:28.684292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:28.684292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:28.684292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:28.684292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:28.685252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:28.685252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:28.685252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:24:28.685252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:24:28.685252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:24:28.685252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:24:28.685252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:24:28.685252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:24:28.685252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:24:28.685252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:28.686381 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 546/1066] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Jun 23 22:24:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:28.690367 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37584bc4-9bbf-4008-af67-1ae422aac30a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:28.716096 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:28.794311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37584bc4-9bbf-4008-af67-1ae422aac30a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:28.828885 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759423af-e508-4196-9185-a62839cd842f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:28.829487 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759423af-e508-4196-9185-a62839cd842f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:28.829778 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759423af-e508-4196-9185-a62839cd842f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:28.830071 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759423af-e508-4196-9185-a62839cd842f None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:28.830909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759423af-e508-4196-9185-a62839cd842f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:28.831261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759423af-e508-4196-9185-a62839cd842f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:28.832842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759423af-e508-4196-9185-a62839cd842f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:28.856199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 521/1067] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Jun 23 22:24:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:28.857465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b9fbc-b761-43d3-9304-b229295cf64c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:28.962847 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b9fbc-b761-43d3-9304-b229295cf64c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:28.980858 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:29.001255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abaa8f9-3385-4890-a227-bd19036dcf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:29.002069 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abaa8f9-3385-4890-a227-bd19036dcf47 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:29.002359 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abaa8f9-3385-4890-a227-bd19036dcf47 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:29.002639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abaa8f9-3385-4890-a227-bd19036dcf47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:29.013737 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abaa8f9-3385-4890-a227-bd19036dcf47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:29.015220 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abaa8f9-3385-4890-a227-bd19036dcf47 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:29.032657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-6abaa8f9-3385-4890-a227-bd19036dcf47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:29.032657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:29.032657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:29.032657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:29.032657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:29.032657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:29.032657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:29.032657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:29.032657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:29.032657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:29.032657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:29.032657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:29.032657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:24:29.032657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:24:29.032657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:24:29.033581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:24:29.033581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:29.033581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:29.033581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:29.033581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:29.033581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:29.033581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:29.033581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:29.033581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:29.033581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:29.033581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:29.033581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:29.033581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:29.033581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:29.033581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:29.033581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:29.033581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:29.034476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:24:29.034476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:24:29.034476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:29.034476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:29.034476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 547/1068] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:29.040025 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759423af-e508-4196-9185-a62839cd842f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:29.111729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6aca4b-24c2-47d4-91b9-8502ac54dc18 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:29.111924 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6aca4b-24c2-47d4-91b9-8502ac54dc18 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:29.112214 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6aca4b-24c2-47d4-91b9-8502ac54dc18 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:29.113805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 548/1069] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:24:29.119277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e454b4-af9a-44bb-8120-c37745b5e7af None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:29.119573 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e454b4-af9a-44bb-8120-c37745b5e7af None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:29.119856 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e454b4-af9a-44bb-8120-c37745b5e7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:29.128186 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:29.140490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-37e454b4-af9a-44bb-8120-c37745b5e7af None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:29.141782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-37e454b4-af9a-44bb-8120-c37745b5e7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:29.151309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759423af-e508-4196-9185-a62839cd842f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:29.186698 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757fdbf-7624-496b-8aa4-51b95f13b65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:29.187395 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757fdbf-7624-496b-8aa4-51b95f13b65b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:29.187684 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757fdbf-7624-496b-8aa4-51b95f13b65b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:29.187967 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757fdbf-7624-496b-8aa4-51b95f13b65b None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:29.188681 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757fdbf-7624-496b-8aa4-51b95f13b65b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:29.188993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757fdbf-7624-496b-8aa4-51b95f13b65b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:29.190402 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757fdbf-7624-496b-8aa4-51b95f13b65b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:29.202461 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 522/1070] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:29.207852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757fdbf-7624-496b-8aa4-51b95f13b65b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:29.214994 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 549/1071] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:29.237218 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e454b4-af9a-44bb-8120-c37745b5e7af None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:29.312604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757fdbf-7624-496b-8aa4-51b95f13b65b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:29.340499 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:29.344229 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e454b4-af9a-44bb-8120-c37745b5e7af None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:29.348250 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e4a63-4233-459c-9abf-3b45ea112071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:29.349617 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e4a63-4233-459c-9abf-3b45ea112071 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:29.349912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e4a63-4233-459c-9abf-3b45ea112071 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:29.350193 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e4a63-4233-459c-9abf-3b45ea112071 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:29.357734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e4a63-4233-459c-9abf-3b45ea112071 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:29.359211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e4a63-4233-459c-9abf-3b45ea112071 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:29.361925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:29.377352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-373e4a63-4233-459c-9abf-3b45ea112071 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:24:29.377352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:29.377352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:29.377352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:29.377352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:29.377352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:29.377352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:29.377352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:29.377352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:29.377352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:29.377352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:29.377352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:29.377352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:24:29.377352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:24:29.377352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:24:29.378325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:24:29.378325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:29.378325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:29.378325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:29.378325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:29.378325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:29.378325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:29.378325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:29.378325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:29.378325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:29.378325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:29.378325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:29.378325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:29.378325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:29.378325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:29.378325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:29.378325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:29.379288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:24:29.379288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:24:29.379288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:24:29.379288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:24:29.379288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:24:29.379288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:24:29.379288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:24:29.379288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:29.379288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 523/1072] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:24:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:29.381975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a26731-28be-49d9-83be-77134b5ee5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:29.382648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a26731-28be-49d9-83be-77134b5ee5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:29.383034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a26731-28be-49d9-83be-77134b5ee5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:29.383253 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a26731-28be-49d9-83be-77134b5ee5a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:29.384648 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e4a63-4233-459c-9abf-3b45ea112071 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:29.457111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a26731-28be-49d9-83be-77134b5ee5a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:29.459176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a26731-28be-49d9-83be-77134b5ee5a2 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:29.498014 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e4a63-4233-459c-9abf-3b45ea112071 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:29.537631 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bb6ea-9658-4481-96ab-4c608d5ec6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:29.538304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bb6ea-9658-4481-96ab-4c608d5ec6ea None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:29.538791 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bb6ea-9658-4481-96ab-4c608d5ec6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:29.539046 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bb6ea-9658-4481-96ab-4c608d5ec6ea None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:29.539872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bb6ea-9658-4481-96ab-4c608d5ec6ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:29.540309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bb6ea-9658-4481-96ab-4c608d5ec6ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:29.541699 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bb6ea-9658-4481-96ab-4c608d5ec6ea None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:29.561115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 524/1073] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:24:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:29.562107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bb6ea-9658-4481-96ab-4c608d5ec6ea None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:29.564754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:29.571658 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 550/1074] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:24:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 22:24:29.576803 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a26731-28be-49d9-83be-77134b5ee5a2 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:29.626218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:29.669917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bb6ea-9658-4481-96ab-4c608d5ec6ea None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:29.681963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a26731-28be-49d9-83be-77134b5ee5a2 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:29.707750 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9c4bb-a82b-4f5f-be1f-69955237d372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:29.708849 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9c4bb-a82b-4f5f-be1f-69955237d372 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:29.709143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9c4bb-a82b-4f5f-be1f-69955237d372 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:29.709430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9c4bb-a82b-4f5f-be1f-69955237d372 None admin] PATH_INFO: `/v3/projects/56da0975019c44409b53be473131143a/users/35d686923ef541819de3c77847d38511/roles/26dbcfd7d83742b1b1cd311915a9c0a8` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:29.711902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9c4bb-a82b-4f5f-be1f-69955237d372 None admin] RBAC: Authorizing `identity:create_grant(project_id=56da0975019c44409b53be473131143a, user_id=35d686923ef541819de3c77847d38511, role_id=26dbcfd7d83742b1b1cd311915a9c0a8)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:29.713331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9c4bb-a82b-4f5f-be1f-69955237d372 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:29.716043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba28b9f-5968-45f7-a9f8-8d036b5d316f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:29.716840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba28b9f-5968-45f7-a9f8-8d036b5d316f None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:29.717131 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba28b9f-5968-45f7-a9f8-8d036b5d316f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:29.717416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba28b9f-5968-45f7-a9f8-8d036b5d316f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:29.728527 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba28b9f-5968-45f7-a9f8-8d036b5d316f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:29.729322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 525/1075] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:24:29 2022] PUT /identity/v3/projects/56da0975019c44409b53be473131143a/users/35d686923ef541819de3c77847d38511/roles/26dbcfd7d83742b1b1cd311915a9c0a8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:24:29.729968 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba28b9f-5968-45f7-a9f8-8d036b5d316f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:29.757863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-9ba28b9f-5968-45f7-a9f8-8d036b5d316f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:29.757863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:29.757863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:29.757863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:29.757863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:29.757863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:29.757863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:29.757863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:29.757863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:29.757863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:29.757863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:29.757863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:29.757863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:24:29.757863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:24:29.757863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:24:29.758885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:24:29.758885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:29.758885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:29.758885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:29.758885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:29.758885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:29.758885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:29.758885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:29.758885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:29.758885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:29.758885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:29.758885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:29.758885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:29.758885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:29.758885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:29.758885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:29.758885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:29.759833 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:24:29.759833 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:24:29.759833 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:29.759833 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:29.759833 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 551/1076] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:29.765520 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9c4bb-a82b-4f5f-be1f-69955237d372 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:29.878572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9c4bb-a82b-4f5f-be1f-69955237d372 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:29.903904 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:29.922406 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc94fd-9d37-4b3c-afbb-b5bafe452a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:29.923133 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc94fd-9d37-4b3c-afbb-b5bafe452a23 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:29.923424 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc94fd-9d37-4b3c-afbb-b5bafe452a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:29.923706 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc94fd-9d37-4b3c-afbb-b5bafe452a23 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:29.924445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc94fd-9d37-4b3c-afbb-b5bafe452a23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:29.924742 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc94fd-9d37-4b3c-afbb-b5bafe452a23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:29.926206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc94fd-9d37-4b3c-afbb-b5bafe452a23 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:29.928343 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25ce10-1f89-43b2-924f-2d48b3ebc956 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:29.929127 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25ce10-1f89-43b2-924f-2d48b3ebc956 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:29.930129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25ce10-1f89-43b2-924f-2d48b3ebc956 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:29.931839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 552/1077] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:24:29.937149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713d582-9b5f-4173-a511-0f3fdd719ea6 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:29.937899 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713d582-9b5f-4173-a511-0f3fdd719ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:29.938191 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713d582-9b5f-4173-a511-0f3fdd719ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:29.940822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 526/1078] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:29.957049 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-3713d582-9b5f-4173-a511-0f3fdd719ea6 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:29.958313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3713d582-9b5f-4173-a511-0f3fdd719ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:30.014259 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:30.039376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 553/1079] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:30.063270 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc94fd-9d37-4b3c-afbb-b5bafe452a23 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:30.182087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc94fd-9d37-4b3c-afbb-b5bafe452a23 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:30.220947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f250c-8541-404a-8362-e4a0db9edb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:30.220947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f250c-8541-404a-8362-e4a0db9edb9d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:30.220947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f250c-8541-404a-8362-e4a0db9edb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:30.221301 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f250c-8541-404a-8362-e4a0db9edb9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:30.233292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f250c-8541-404a-8362-e4a0db9edb9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:24:30.241059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b51f250c-8541-404a-8362-e4a0db9edb9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:24:30.241059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:30.241059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:30.241059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:30.241059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:30.241059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:30.241059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:30.241059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:30.241059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:30.241059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:30.241059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:30.241059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:30.241059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:24:30.241059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:24:30.241059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:24:30.241059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:24:30.242153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:24:30.242153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:24:30.242153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:24:30.242153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:24:30.242153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:30.242153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:30.242153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:30.242153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:30.242153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:30.242153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:30.242153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:30.242153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:30.242153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:30.242153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:30.242153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:30.242153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:30.243262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:30.243262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:30.243262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:30.243262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:30.243262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:24:30.243262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:24:30.243262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:24:30.243262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:24:30.243262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:24:30.243262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:24:30.243262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:24:30.243262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:30.243262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 527/1080] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:24:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:30.248942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713d582-9b5f-4173-a511-0f3fdd719ea6 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:30.345404 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713d582-9b5f-4173-a511-0f3fdd719ea6 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:30.378850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218df757-b74e-4bfc-9d86-d7e455cc1070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:30.379543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218df757-b74e-4bfc-9d86-d7e455cc1070 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:30.379830 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218df757-b74e-4bfc-9d86-d7e455cc1070 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:30.380110 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218df757-b74e-4bfc-9d86-d7e455cc1070 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:30.381616 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218df757-b74e-4bfc-9d86-d7e455cc1070 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:30.381927 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218df757-b74e-4bfc-9d86-d7e455cc1070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:30.383334 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218df757-b74e-4bfc-9d86-d7e455cc1070 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:30.391411 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 554/1081] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:24:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:30.397148 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f250c-8541-404a-8362-e4a0db9edb9d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:30.440101 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:30.500665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f250c-8541-404a-8362-e4a0db9edb9d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:30.535204 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46978313-1aa2-474c-94e0-68bf38a0bb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:30.536078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46978313-1aa2-474c-94e0-68bf38a0bb42 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:30.536370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46978313-1aa2-474c-94e0-68bf38a0bb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:30.536698 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46978313-1aa2-474c-94e0-68bf38a0bb42 None admin] PATH_INFO: `/v3/users/399341e6554244a297ccce389c17bc09` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:30.537966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46978313-1aa2-474c-94e0-68bf38a0bb42 None admin] RBAC: Authorizing `identity:get_user(user_id=399341e6554244a297ccce389c17bc09)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:30.539432 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46978313-1aa2-474c-94e0-68bf38a0bb42 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:30.541065 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 528/1082] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:24:30 2022] GET /identity/v3/users/399341e6554244a297ccce389c17bc09 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:30.546745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218df757-b74e-4bfc-9d86-d7e455cc1070 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:30.591925 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:30.651718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218df757-b74e-4bfc-9d86-d7e455cc1070 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:30.683831 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d8feb-969b-4b1b-8865-801f3aa0f541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:30.684692 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d8feb-969b-4b1b-8865-801f3aa0f541 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:30.684980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d8feb-969b-4b1b-8865-801f3aa0f541 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:30.685259 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d8feb-969b-4b1b-8865-801f3aa0f541 None admin] PATH_INFO: `/v3/projects/c4a7b621f2b149eb9e972bb032e21bda` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:30.686430 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59d8feb-969b-4b1b-8865-801f3aa0f541 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a7b621f2b149eb9e972bb032e21bda)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:30.687886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59d8feb-969b-4b1b-8865-801f3aa0f541 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:30.689573 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 555/1083] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:24:30 2022] GET /identity/v3/projects/c4a7b621f2b149eb9e972bb032e21bda => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:30.695953 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46978313-1aa2-474c-94e0-68bf38a0bb42 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:30.791588 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46978313-1aa2-474c-94e0-68bf38a0bb42 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:30.799976 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:30.824842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7609a-743f-43b7-9bbf-f730e67a8424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:30.825728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7609a-743f-43b7-9bbf-f730e67a8424 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:30.826011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7609a-743f-43b7-9bbf-f730e67a8424 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:30.826287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7609a-743f-43b7-9bbf-f730e67a8424 None admin] PATH_INFO: `/v3/projects/c4a7b621f2b149eb9e972bb032e21bda/users/399341e6554244a297ccce389c17bc09/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:30.828238 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7609a-743f-43b7-9bbf-f730e67a8424 None admin] RBAC: Authorizing `identity:create_grant(project_id=c4a7b621f2b149eb9e972bb032e21bda, user_id=399341e6554244a297ccce389c17bc09, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:30.829651 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7609a-743f-43b7-9bbf-f730e67a8424 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:30.843849 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 529/1084] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:24:30 2022] PUT /identity/v3/projects/c4a7b621f2b149eb9e972bb032e21bda/users/399341e6554244a297ccce389c17bc09/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:24:31.808230 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad549e0-4326-4dde-b066-e0680d81a2cc None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:31.808779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad549e0-4326-4dde-b066-e0680d81a2cc None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:31.808779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad549e0-4326-4dde-b066-e0680d81a2cc None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:31.810143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 556/1085] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:31.815445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4eed1e-87b0-4cc3-831f-20f019e53450 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:31.815840 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4eed1e-87b0-4cc3-831f-20f019e53450 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:31.816140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4eed1e-87b0-4cc3-831f-20f019e53450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:31.833352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3f4eed1e-87b0-4cc3-831f-20f019e53450 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:31.834785 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3f4eed1e-87b0-4cc3-831f-20f019e53450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:31.901076 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 530/1086] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:31.925232 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad549e0-4326-4dde-b066-e0680d81a2cc None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:32.029763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad549e0-4326-4dde-b066-e0680d81a2cc None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:32.042166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:32.062985 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986c8c7-ff81-4735-a7e0-0fc8a4ca528f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:32.063711 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986c8c7-ff81-4735-a7e0-0fc8a4ca528f None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:32.064000 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986c8c7-ff81-4735-a7e0-0fc8a4ca528f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:32.064280 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986c8c7-ff81-4735-a7e0-0fc8a4ca528f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:32.075151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a986c8c7-ff81-4735-a7e0-0fc8a4ca528f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:24:32.083154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-a986c8c7-ff81-4735-a7e0-0fc8a4ca528f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:24:32.083154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:32.083154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:32.083154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:32.083154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:32.083154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:32.083154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:32.083154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:32.083154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:32.083154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:32.083154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:32.083154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:32.083154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:24:32.083154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:24:32.083154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:24:32.083154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:24:32.089795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:24:32.089795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:24:32.089795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:24:32.089795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:24:32.089795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:32.089795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:32.089795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:32.089795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:32.089795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:32.089795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:32.089795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:32.089795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:32.089795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:32.089795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:32.089795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:32.089795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:32.090912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:32.090912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:32.090912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:32.090912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:32.090912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:24:32.090912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:24:32.090912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:24:32.090912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:24:32.090912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:24:32.090912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:24:32.090912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 22:24:32.090912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:32.090912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 557/1087] 10.222.0.43 () {56 vars in 1120 bytes} [Thu Jun 23 22:24:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:32.092118 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4eed1e-87b0-4cc3-831f-20f019e53450 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:32.189825 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:32.193833 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4eed1e-87b0-4cc3-831f-20f019e53450 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:32.229452 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca951245-6b34-4b09-81b2-12dfed2ef575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:32.230118 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca951245-6b34-4b09-81b2-12dfed2ef575 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:32.230406 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca951245-6b34-4b09-81b2-12dfed2ef575 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:32.230688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca951245-6b34-4b09-81b2-12dfed2ef575 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:32.231466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca951245-6b34-4b09-81b2-12dfed2ef575 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:32.231784 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca951245-6b34-4b09-81b2-12dfed2ef575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:32.233139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca951245-6b34-4b09-81b2-12dfed2ef575 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:32.241565 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 531/1088] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Jun 23 22:24:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:32.247162 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986c8c7-ff81-4735-a7e0-0fc8a4ca528f None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:32.345037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986c8c7-ff81-4735-a7e0-0fc8a4ca528f None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:32.380237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e76c8a-eb84-42ee-9d5d-ac17a243bebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:32.382653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e76c8a-eb84-42ee-9d5d-ac17a243bebd None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:32.382653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e76c8a-eb84-42ee-9d5d-ac17a243bebd None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:32.382653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e76c8a-eb84-42ee-9d5d-ac17a243bebd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:32.391814 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e76c8a-eb84-42ee-9d5d-ac17a243bebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:32.393260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e76c8a-eb84-42ee-9d5d-ac17a243bebd None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:32.411240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-62e76c8a-eb84-42ee-9d5d-ac17a243bebd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:32.411240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:32.411240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:32.411240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:32.411240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:32.411240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:32.411240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:32.411240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:32.411240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:32.411240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:32.411240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:32.411240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:32.411240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:24:32.411240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:24:32.411240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:24:32.412410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:24:32.412410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:32.412410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:32.412410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:32.412410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:32.412410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:32.412410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:32.412410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:32.412410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:32.412410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:32.412410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:32.412410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:32.412410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:32.412410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:32.412410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:32.412410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:32.412410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:32.419470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:24:32.419470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:24:32.419470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:32.419470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:32.419470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 558/1089] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:32.419978 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca951245-6b34-4b09-81b2-12dfed2ef575 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:32.516549 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca951245-6b34-4b09-81b2-12dfed2ef575 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:32.550015 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e40f3-d0d6-4c29-b23a-ee70c661c83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:32.550678 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e40f3-d0d6-4c29-b23a-ee70c661c83d None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:32.550992 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e40f3-d0d6-4c29-b23a-ee70c661c83d None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:32.551277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e40f3-d0d6-4c29-b23a-ee70c661c83d None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:32.552042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e40f3-d0d6-4c29-b23a-ee70c661c83d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:32.552357 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e40f3-d0d6-4c29-b23a-ee70c661c83d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:32.553728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e40f3-d0d6-4c29-b23a-ee70c661c83d None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:32.565300 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 532/1090] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:32.571288 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e76c8a-eb84-42ee-9d5d-ac17a243bebd None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:32.597014 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:32.671407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e76c8a-eb84-42ee-9d5d-ac17a243bebd None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:32.704477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dad806-187b-4259-ae2a-4eadadb64322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:32.705283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dad806-187b-4259-ae2a-4eadadb64322 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:32.705576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dad806-187b-4259-ae2a-4eadadb64322 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:32.705862 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dad806-187b-4259-ae2a-4eadadb64322 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:32.714773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dad806-187b-4259-ae2a-4eadadb64322 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:32.716263 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dad806-187b-4259-ae2a-4eadadb64322 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:32.730576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-f5dad806-187b-4259-ae2a-4eadadb64322 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:24:32.730576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:32.730576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:32.730576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:32.730576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:32.730576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:32.730576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:32.730576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:32.730576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:32.730576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:32.730576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:32.730576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:32.730576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:24:32.730576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:24:32.730576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:24:32.730576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:24:32.731628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:32.731628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:32.731628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:24:32.731628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:24:32.731628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:24:32.731628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:24:32.731628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:32.731628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:32.731628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:32.731628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:32.731628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:32.731628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:32.731628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:32.731628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:32.731628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:32.731628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:32.731628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:32.731628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:32.732587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:32.732587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:32.732587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:24:32.732587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:24:32.732587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:24:32.732587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:24:32.732587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:24:32.732587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:24:32.732587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 22:24:32.732587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:32.732587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 559/1091] 10.222.0.43 () {56 vars in 1093 bytes} [Thu Jun 23 22:24:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:32.737723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e40f3-d0d6-4c29-b23a-ee70c661c83d None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:32.760293 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:32.839292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e40f3-d0d6-4c29-b23a-ee70c661c83d None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:32.861530 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a8c60-1b53-4f6c-b57c-b57047803155 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:32.862017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a8c60-1b53-4f6c-b57c-b57047803155 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:32.862017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a8c60-1b53-4f6c-b57c-b57047803155 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:32.863422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 560/1092] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:24:32.869100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df1b45-58b4-409e-998f-15c54e630782 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:32.869387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df1b45-58b4-409e-998f-15c54e630782 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:32.869924 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df1b45-58b4-409e-998f-15c54e630782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:32.874337 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce196ed-e54f-40b6-8c75-08ea4e43afd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:32.875042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce196ed-e54f-40b6-8c75-08ea4e43afd6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:32.875334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce196ed-e54f-40b6-8c75-08ea4e43afd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:32.875621 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce196ed-e54f-40b6-8c75-08ea4e43afd6 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:32.876459 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce196ed-e54f-40b6-8c75-08ea4e43afd6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:32.876772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce196ed-e54f-40b6-8c75-08ea4e43afd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:32.878186 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce196ed-e54f-40b6-8c75-08ea4e43afd6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:32.886340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-39df1b45-58b4-409e-998f-15c54e630782 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:32.887562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-39df1b45-58b4-409e-998f-15c54e630782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:32.895145 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 533/1093] 10.222.0.43 () {56 vars in 1133 bytes} [Thu Jun 23 22:24:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:32.901233 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce196ed-e54f-40b6-8c75-08ea4e43afd6 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:32.954409 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 561/1094] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:32.978550 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df1b45-58b4-409e-998f-15c54e630782 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:32.980801 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:32.993903 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:33.002149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce196ed-e54f-40b6-8c75-08ea4e43afd6 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:33.037963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d031c-f845-49b3-87d2-be6f2b14b676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:33.038609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d031c-f845-49b3-87d2-be6f2b14b676 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:33.038894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d031c-f845-49b3-87d2-be6f2b14b676 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:33.039199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d031c-f845-49b3-87d2-be6f2b14b676 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:33.079614 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df1b45-58b4-409e-998f-15c54e630782 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:33.106225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d031c-f845-49b3-87d2-be6f2b14b676 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:33.107708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d031c-f845-49b3-87d2-be6f2b14b676 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:33.113591 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d913d68-60f4-40f4-9b9b-5f7cf91b4cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:33.114326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d913d68-60f4-40f4-9b9b-5f7cf91b4cbf None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:33.114660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d913d68-60f4-40f4-9b9b-5f7cf91b4cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:33.115029 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d913d68-60f4-40f4-9b9b-5f7cf91b4cbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:33.127790 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d913d68-60f4-40f4-9b9b-5f7cf91b4cbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:24:33.135462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-8d913d68-60f4-40f4-9b9b-5f7cf91b4cbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:24:33.135462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:33.135462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:33.135462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:33.135462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:33.135462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:33.135462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:33.135462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:33.135462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:33.135462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:33.135462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:33.135462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:33.135462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:24:33.135462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:24:33.135462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:24:33.135462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:24:33.136535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:24:33.136535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:24:33.136535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:24:33.136535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:24:33.136535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:33.136535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:33.136535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:33.136535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:33.136535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:33.136535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:33.136535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:33.136535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:33.136535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:33.136535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:33.136535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:33.136535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:33.137396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:33.137396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:33.137396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:33.137396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:33.137396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:24:33.137396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:24:33.137396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:24:33.137396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:24:33.137396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:24:33.137396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:24:33.137396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:24:33.137396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:33.137396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 562/1095] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:24:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:33.142580 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d913d68-60f4-40f4-9b9b-5f7cf91b4cbf None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:33.190116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:33.213392 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 534/1096] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:24:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 22:24:33.217379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:33.219224 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d031c-f845-49b3-87d2-be6f2b14b676 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:33.243368 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d913d68-60f4-40f4-9b9b-5f7cf91b4cbf None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:33.277657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25a91b-1545-4f13-bd59-b528f965d742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:33.278333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25a91b-1545-4f13-bd59-b528f965d742 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:33.278624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25a91b-1545-4f13-bd59-b528f965d742 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:33.278909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25a91b-1545-4f13-bd59-b528f965d742 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:33.279679 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25a91b-1545-4f13-bd59-b528f965d742 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:33.279993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25a91b-1545-4f13-bd59-b528f965d742 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:33.281444 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25a91b-1545-4f13-bd59-b528f965d742 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:33.289439 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 563/1097] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:24:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:33.295187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25a91b-1545-4f13-bd59-b528f965d742 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:33.316898 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d031c-f845-49b3-87d2-be6f2b14b676 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:33.352173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5e557-b1f5-4784-99a5-3adbe53845bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:33.353068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5e557-b1f5-4784-99a5-3adbe53845bf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:33.353424 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5e557-b1f5-4784-99a5-3adbe53845bf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:33.353716 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5e557-b1f5-4784-99a5-3adbe53845bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:33.363572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5e557-b1f5-4784-99a5-3adbe53845bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:33.366241 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5e557-b1f5-4784-99a5-3adbe53845bf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:33.385171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-65f5e557-b1f5-4784-99a5-3adbe53845bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:33.385171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:33.385171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:33.385171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:33.385171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:33.385171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:33.385171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:33.385171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:33.385171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:33.385171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:33.385171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:33.385171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:33.385171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:24:33.385171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:24:33.385171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:24:33.386121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:24:33.386121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:33.386121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:33.386121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:33.386121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:33.386121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:33.386121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:33.386121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:33.386121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:33.386121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:33.386121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:33.386121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:33.386121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:33.386121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:33.386121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:33.386121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:33.386121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:33.387284 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:24:33.387284 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:24:33.387284 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:24:33.387284 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:33.387284 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 535/1098] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:33.391860 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5e557-b1f5-4784-99a5-3adbe53845bf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:33.393127 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25a91b-1545-4f13-bd59-b528f965d742 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:33.422016 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:33.428092 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0042987-0219-4144-9c76-18d9314a6c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:33.428955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0042987-0219-4144-9c76-18d9314a6c1b None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:33.429264 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0042987-0219-4144-9c76-18d9314a6c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:33.429543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0042987-0219-4144-9c76-18d9314a6c1b None admin] PATH_INFO: `/v3/users/399341e6554244a297ccce389c17bc09` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:33.430798 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0042987-0219-4144-9c76-18d9314a6c1b None admin] RBAC: Authorizing `identity:get_user(user_id=399341e6554244a297ccce389c17bc09)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:33.432271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0042987-0219-4144-9c76-18d9314a6c1b None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:33.433942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 564/1099] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:24:33 2022] GET /identity/v3/users/399341e6554244a297ccce389c17bc09 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:33.439635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0042987-0219-4144-9c76-18d9314a6c1b None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:33.443208 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:33.494379 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5e557-b1f5-4784-99a5-3adbe53845bf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:33.528719 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef69f0-8878-4e79-8993-96cb7b87f191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:33.529381 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef69f0-8878-4e79-8993-96cb7b87f191 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:33.529675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef69f0-8878-4e79-8993-96cb7b87f191 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:33.529961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef69f0-8878-4e79-8993-96cb7b87f191 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:33.530681 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef69f0-8878-4e79-8993-96cb7b87f191 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:33.531031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef69f0-8878-4e79-8993-96cb7b87f191 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:33.532413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef69f0-8878-4e79-8993-96cb7b87f191 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:33.538853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0042987-0219-4144-9c76-18d9314a6c1b None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:33.543781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 536/1100] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:33.549269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef69f0-8878-4e79-8993-96cb7b87f191 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:33.572262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a58b3-271a-4638-9f71-0751203b0dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:33.572919 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a58b3-271a-4638-9f71-0751203b0dff None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:33.573210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a58b3-271a-4638-9f71-0751203b0dff None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:33.573494 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a58b3-271a-4638-9f71-0751203b0dff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:33.639462 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:33.639691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a58b3-271a-4638-9f71-0751203b0dff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:33.641200 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a58b3-271a-4638-9f71-0751203b0dff None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:33.643811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:33.650673 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef69f0-8878-4e79-8993-96cb7b87f191 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:33.685163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9958935-02da-483e-bea2-e3699389f6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:33.685960 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9958935-02da-483e-bea2-e3699389f6aa None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:33.686246 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9958935-02da-483e-bea2-e3699389f6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:33.686527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9958935-02da-483e-bea2-e3699389f6aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:33.693460 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9958935-02da-483e-bea2-e3699389f6aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:33.694891 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9958935-02da-483e-bea2-e3699389f6aa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:33.709069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-b9958935-02da-483e-bea2-e3699389f6aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:24:33.709069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:33.709069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:33.709069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:33.709069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:33.709069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:33.709069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:33.709069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:33.709069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:33.709069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:33.709069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:33.709069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:33.709069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:24:33.709069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:24:33.709069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:24:33.709977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:24:33.709977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:33.709977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:33.709977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:33.709977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:33.709977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:33.709977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:33.709977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:33.709977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:33.709977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:33.709977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:33.709977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:33.709977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:33.709977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:33.709977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:33.709977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:33.709977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:33.711108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:24:33.711108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:24:33.711108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:24:33.711108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:24:33.711108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:24:33.711108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:24:33.711108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:24:33.711108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:33.711108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 537/1101] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:24:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:24:33.715954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9958935-02da-483e-bea2-e3699389f6aa None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:33.747649 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 565/1102] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:24:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 22:24:33.748868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a58b3-271a-4638-9f71-0751203b0dff None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:33.814410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9958935-02da-483e-bea2-e3699389f6aa None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:33.845391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a58b3-271a-4638-9f71-0751203b0dff None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:33.847741 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:33.849820 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c86b37-8188-4c1c-9118-dd7ad2d29b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:33.850529 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c86b37-8188-4c1c-9118-dd7ad2d29b3b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:33.850813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c86b37-8188-4c1c-9118-dd7ad2d29b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:33.851125 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c86b37-8188-4c1c-9118-dd7ad2d29b3b None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:33.851914 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c86b37-8188-4c1c-9118-dd7ad2d29b3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:33.852236 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c86b37-8188-4c1c-9118-dd7ad2d29b3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:33.853666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c86b37-8188-4c1c-9118-dd7ad2d29b3b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:33.871089 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 538/1103] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:24:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:33.872212 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c86b37-8188-4c1c-9118-dd7ad2d29b3b None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:33.881695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe81891-a7fd-4a2e-8365-896dbd12e710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:33.882566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe81891-a7fd-4a2e-8365-896dbd12e710 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:33.882851 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe81891-a7fd-4a2e-8365-896dbd12e710 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:33.883165 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe81891-a7fd-4a2e-8365-896dbd12e710 None admin] PATH_INFO: `/v3/projects/c4a7b621f2b149eb9e972bb032e21bda` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:33.884355 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe81891-a7fd-4a2e-8365-896dbd12e710 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a7b621f2b149eb9e972bb032e21bda)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:33.884808 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:33.885898 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe81891-a7fd-4a2e-8365-896dbd12e710 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:33.892542 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 566/1104] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:24:33 2022] GET /identity/v3/projects/c4a7b621f2b149eb9e972bb032e21bda => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:33.893325 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe81891-a7fd-4a2e-8365-896dbd12e710 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:33.973726 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c86b37-8188-4c1c-9118-dd7ad2d29b3b None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:33.993487 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe81891-a7fd-4a2e-8365-896dbd12e710 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:34.007942 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d97dff-b96b-4565-aa17-ec84c533e857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:34.008595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d97dff-b96b-4565-aa17-ec84c533e857 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:34.008886 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d97dff-b96b-4565-aa17-ec84c533e857 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:34.009167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d97dff-b96b-4565-aa17-ec84c533e857 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:34.009984 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d97dff-b96b-4565-aa17-ec84c533e857 None admin] RBAC: Adding query filter params (role.id=26dbcfd7d83742b1b1cd311915a9c0a8, scope.project.id=56da0975019c44409b53be473131143a, user.id=35d686923ef541819de3c77847d38511) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:34.010294 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d97dff-b96b-4565-aa17-ec84c533e857 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:34.011736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d97dff-b96b-4565-aa17-ec84c533e857 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:34.017483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 539/1105] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:24:33 2022] GET /identity/v3/role_assignments?user.id=35d686923ef541819de3c77847d38511&scope.project.id=56da0975019c44409b53be473131143a&role.id=26dbcfd7d83742b1b1cd311915a9c0a8 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:34.030813 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2f1a6-f9b4-4f98-8df3-56ecd06c732b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:34.031527 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2f1a6-f9b4-4f98-8df3-56ecd06c732b None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:34.031815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2f1a6-f9b4-4f98-8df3-56ecd06c732b None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:34.032104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2f1a6-f9b4-4f98-8df3-56ecd06c732b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:34.032961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2f1a6-f9b4-4f98-8df3-56ecd06c732b None admin] RBAC: Adding query filter params (role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.project.id=c4a7b621f2b149eb9e972bb032e21bda, user.id=399341e6554244a297ccce389c17bc09) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:34.033283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2f1a6-f9b4-4f98-8df3-56ecd06c732b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:34.034785 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2f1a6-f9b4-4f98-8df3-56ecd06c732b None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:34.042345 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 567/1106] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:24:33 2022] GET /identity/v3/role_assignments?user.id=399341e6554244a297ccce389c17bc09&scope.project.id=c4a7b621f2b149eb9e972bb032e21bda&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:36.142338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c98e7c-c22a-4f62-a7bd-2b1702261958 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:36.142761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c98e7c-c22a-4f62-a7bd-2b1702261958 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:36.142904 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c98e7c-c22a-4f62-a7bd-2b1702261958 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:36.144373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 540/1107] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:36.150166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb046ac-ae29-409d-a20f-e3c18f7ec4cc None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:36.150584 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb046ac-ae29-409d-a20f-e3c18f7ec4cc None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:36.150731 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb046ac-ae29-409d-a20f-e3c18f7ec4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:36.167039 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-4eb046ac-ae29-409d-a20f-e3c18f7ec4cc None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:36.168205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-4eb046ac-ae29-409d-a20f-e3c18f7ec4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:36.196322 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:36.232872 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 568/1108] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:36.258197 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c98e7c-c22a-4f62-a7bd-2b1702261958 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:36.332792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:36.358284 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c98e7c-c22a-4f62-a7bd-2b1702261958 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:36.394019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eb982d-9e1d-4780-8d5c-9ea3cac2e5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:36.394744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb982d-9e1d-4780-8d5c-9ea3cac2e5ca None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:36.395061 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb982d-9e1d-4780-8d5c-9ea3cac2e5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:36.395342 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb982d-9e1d-4780-8d5c-9ea3cac2e5ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:36.397043 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb982d-9e1d-4780-8d5c-9ea3cac2e5ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:36.398721 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eb982d-9e1d-4780-8d5c-9ea3cac2e5ca None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:36.400363 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 541/1109] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:36.406196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb046ac-ae29-409d-a20f-e3c18f7ec4cc None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:36.503826 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb046ac-ae29-409d-a20f-e3c18f7ec4cc None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:36.536929 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd848f-8f38-415c-9def-550d9cb46164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:36.537607 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd848f-8f38-415c-9def-550d9cb46164 None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:36.537894 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd848f-8f38-415c-9def-550d9cb46164 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:36.538174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd848f-8f38-415c-9def-550d9cb46164 None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:36.538884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd848f-8f38-415c-9def-550d9cb46164 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:36.540353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd848f-8f38-415c-9def-550d9cb46164 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:36.559530 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 569/1110] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 22:24:36 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:24:38.615806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b669975-185a-4a42-afe6-56f888da82b6 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:38.616338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b669975-185a-4a42-afe6-56f888da82b6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:38.616451 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b669975-185a-4a42-afe6-56f888da82b6 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:38.617731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 542/1111] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:38.622745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f31b27-55a6-46b9-a0a1-16d79eae0068 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:38.623206 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f31b27-55a6-46b9-a0a1-16d79eae0068 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:38.623345 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f31b27-55a6-46b9-a0a1-16d79eae0068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:38.639765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-70f31b27-55a6-46b9-a0a1-16d79eae0068 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:38.640943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-70f31b27-55a6-46b9-a0a1-16d79eae0068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:38.657796 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:38.707869 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 570/1112] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:38.732243 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b669975-185a-4a42-afe6-56f888da82b6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:38.831828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b669975-185a-4a42-afe6-56f888da82b6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:38.864277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f05fca-4a39-4b76-8928-a2cc5ddb164c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:38.866428 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f05fca-4a39-4b76-8928-a2cc5ddb164c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:38.866650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f05fca-4a39-4b76-8928-a2cc5ddb164c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:38.866956 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f05fca-4a39-4b76-8928-a2cc5ddb164c None admin] PATH_INFO: `/v3/roles/61397d4a1ccd4a8ca479f98186ed489b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:38.868438 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f05fca-4a39-4b76-8928-a2cc5ddb164c None admin] RBAC: Authorizing `identity:get_role(role_id=61397d4a1ccd4a8ca479f98186ed489b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:38.869792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f05fca-4a39-4b76-8928-a2cc5ddb164c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:38.871203 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 543/1113] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:24:38 2022] GET /identity/v3/roles/61397d4a1ccd4a8ca479f98186ed489b => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:38.876884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f31b27-55a6-46b9-a0a1-16d79eae0068 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:38.973205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f31b27-55a6-46b9-a0a1-16d79eae0068 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:39.006665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af6581-5827-4622-9f88-cbb0a0dc79d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:39.007551 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af6581-5827-4622-9f88-cbb0a0dc79d1 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:39.007841 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af6581-5827-4622-9f88-cbb0a0dc79d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:39.008122 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af6581-5827-4622-9f88-cbb0a0dc79d1 None admin] PATH_INFO: `/v3/users/dac15ae97fd34a718327e4c09f36c33c` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:39.020609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98af6581-5827-4622-9f88-cbb0a0dc79d1 None admin] RBAC: Authorizing `identity:get_user(user_id=dac15ae97fd34a718327e4c09f36c33c)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:39.022155 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98af6581-5827-4622-9f88-cbb0a0dc79d1 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:39.024182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 571/1114] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:24:38 2022] GET /identity/v3/users/dac15ae97fd34a718327e4c09f36c33c => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:39.029741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f05fca-4a39-4b76-8928-a2cc5ddb164c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:39.042187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:39.128182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f05fca-4a39-4b76-8928-a2cc5ddb164c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:39.163383 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910967f-c6be-4b1d-8962-1da600179052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:39.164050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910967f-c6be-4b1d-8962-1da600179052 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:39.164342 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910967f-c6be-4b1d-8962-1da600179052 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:39.164626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910967f-c6be-4b1d-8962-1da600179052 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:39.230309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910967f-c6be-4b1d-8962-1da600179052 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:39.231775 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910967f-c6be-4b1d-8962-1da600179052 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:39.242642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:39.328325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 544/1115] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:24:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:24:39.335180 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af6581-5827-4622-9f88-cbb0a0dc79d1 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:39.347036 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:39.435850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af6581-5827-4622-9f88-cbb0a0dc79d1 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:39.469155 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af14423-29c5-4e0b-8cdc-7e946894f2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:39.470016 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af14423-29c5-4e0b-8cdc-7e946894f2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:39.470302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af14423-29c5-4e0b-8cdc-7e946894f2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:39.470581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af14423-29c5-4e0b-8cdc-7e946894f2c9 None admin] PATH_INFO: `/v3/projects/c4a7b621f2b149eb9e972bb032e21bda` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:39.471783 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af14423-29c5-4e0b-8cdc-7e946894f2c9 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a7b621f2b149eb9e972bb032e21bda)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:39.473216 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af14423-29c5-4e0b-8cdc-7e946894f2c9 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:39.475976 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 572/1116] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:24:39 2022] GET /identity/v3/projects/c4a7b621f2b149eb9e972bb032e21bda => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:39.482031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910967f-c6be-4b1d-8962-1da600179052 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:39.578599 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910967f-c6be-4b1d-8962-1da600179052 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:39.588867 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:39.612657 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b187fd-f23c-4151-ac47-b7a5387818d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:39.613305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b187fd-f23c-4151-ac47-b7a5387818d1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:39.613592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b187fd-f23c-4151-ac47-b7a5387818d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:39.613874 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b187fd-f23c-4151-ac47-b7a5387818d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:39.615272 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b187fd-f23c-4151-ac47-b7a5387818d1 None admin] RBAC: Adding query filter params (role.id=61397d4a1ccd4a8ca479f98186ed489b, scope.project.id=c4a7b621f2b149eb9e972bb032e21bda, user.id=dac15ae97fd34a718327e4c09f36c33c) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:39.615676 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b187fd-f23c-4151-ac47-b7a5387818d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:39.617083 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b187fd-f23c-4151-ac47-b7a5387818d1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:39.622399 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 545/1117] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:24:39 2022] GET /identity/v3/role_assignments?user.id=dac15ae97fd34a718327e4c09f36c33c&scope.project.id=c4a7b621f2b149eb9e972bb032e21bda&role.id=61397d4a1ccd4a8ca479f98186ed489b => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:41.656456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca350e12-cf44-4954-b0cf-b8913909b1fe None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:41.656985 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca350e12-cf44-4954-b0cf-b8913909b1fe None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:41.657338 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca350e12-cf44-4954-b0cf-b8913909b1fe None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:41.658766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 573/1118] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:41.665833 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcceda2-e0da-43c7-9e94-d160c627bc29 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:41.670777 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcceda2-e0da-43c7-9e94-d160c627bc29 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:41.670777 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcceda2-e0da-43c7-9e94-d160c627bc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:41.685093 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8bcceda2-e0da-43c7-9e94-d160c627bc29 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:41.686303 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8bcceda2-e0da-43c7-9e94-d160c627bc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:41.760732 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 546/1119] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:41.784012 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca350e12-cf44-4954-b0cf-b8913909b1fe None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:41.868526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:41.893094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca350e12-cf44-4954-b0cf-b8913909b1fe None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:41.929768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e505a-93ec-417a-b56e-4bea7d84f5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:41.930543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e505a-93ec-417a-b56e-4bea7d84f5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:41.930827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e505a-93ec-417a-b56e-4bea7d84f5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:41.931138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e505a-93ec-417a-b56e-4bea7d84f5f5 None admin] PATH_INFO: `/v3/roles/61397d4a1ccd4a8ca479f98186ed489b` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:41.932625 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e505a-93ec-417a-b56e-4bea7d84f5f5 None admin] RBAC: Authorizing `identity:get_role(role_id=61397d4a1ccd4a8ca479f98186ed489b)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:41.933996 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e505a-93ec-417a-b56e-4bea7d84f5f5 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:41.935381 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 574/1120] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:24:41 2022] GET /identity/v3/roles/61397d4a1ccd4a8ca479f98186ed489b => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:41.941241 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcceda2-e0da-43c7-9e94-d160c627bc29 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:42.054075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcceda2-e0da-43c7-9e94-d160c627bc29 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:42.056484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:42.091952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed4aff-db49-4d97-bf2d-9ddbaf33d7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:42.092823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed4aff-db49-4d97-bf2d-9ddbaf33d7e1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:42.093114 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed4aff-db49-4d97-bf2d-9ddbaf33d7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:42.093399 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed4aff-db49-4d97-bf2d-9ddbaf33d7e1 None admin] PATH_INFO: `/v3/users/dac15ae97fd34a718327e4c09f36c33c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:42.094607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed4aff-db49-4d97-bf2d-9ddbaf33d7e1 None admin] RBAC: Authorizing `identity:get_user(user_id=dac15ae97fd34a718327e4c09f36c33c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:42.096088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed4aff-db49-4d97-bf2d-9ddbaf33d7e1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:42.097743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 547/1121] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:24:41 2022] GET /identity/v3/users/dac15ae97fd34a718327e4c09f36c33c => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:42.105952 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e505a-93ec-417a-b56e-4bea7d84f5f5 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:42.241773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e505a-93ec-417a-b56e-4bea7d84f5f5 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:42.292848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd2a7b-4865-416f-8197-45ceded44404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:42.293717 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd2a7b-4865-416f-8197-45ceded44404 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:42.294014 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd2a7b-4865-416f-8197-45ceded44404 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:42.294293 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd2a7b-4865-416f-8197-45ceded44404 None admin] PATH_INFO: `/v3/projects/c4a7b621f2b149eb9e972bb032e21bda` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:42.295499 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd2a7b-4865-416f-8197-45ceded44404 None admin] RBAC: Authorizing `identity:get_project(project_id=c4a7b621f2b149eb9e972bb032e21bda)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:42.296947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd2a7b-4865-416f-8197-45ceded44404 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:42.298638 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 575/1122] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:24:42 2022] GET /identity/v3/projects/c4a7b621f2b149eb9e972bb032e21bda => generated 311 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:42.304573 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed4aff-db49-4d97-bf2d-9ddbaf33d7e1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:42.403191 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed4aff-db49-4d97-bf2d-9ddbaf33d7e1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:42.436287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779ba7f-b37d-49e8-b2f8-71887802481a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:42.437202 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779ba7f-b37d-49e8-b2f8-71887802481a None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:42.437491 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779ba7f-b37d-49e8-b2f8-71887802481a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:42.437779 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779ba7f-b37d-49e8-b2f8-71887802481a None admin] PATH_INFO: `/v3/projects/c4a7b621f2b149eb9e972bb032e21bda/users/dac15ae97fd34a718327e4c09f36c33c/roles/61397d4a1ccd4a8ca479f98186ed489b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:42.439993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779ba7f-b37d-49e8-b2f8-71887802481a None admin] RBAC: Authorizing `identity:create_grant(project_id=c4a7b621f2b149eb9e972bb032e21bda, user_id=dac15ae97fd34a718327e4c09f36c33c, role_id=61397d4a1ccd4a8ca479f98186ed489b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:42.441409 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779ba7f-b37d-49e8-b2f8-71887802481a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:42.454080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 548/1123] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:24:42 2022] PUT /identity/v3/projects/c4a7b621f2b149eb9e972bb032e21bda/users/dac15ae97fd34a718327e4c09f36c33c/roles/61397d4a1ccd4a8ca479f98186ed489b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:24:44.437932 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd374b-ef1d-4892-8183-b93d1d67973e None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:44.438436 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd374b-ef1d-4892-8183-b93d1d67973e None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:44.438436 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd374b-ef1d-4892-8183-b93d1d67973e None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:44.439841 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 576/1124] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:44.445673 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524ea3f-07e6-4a2d-8248-6496c2aa162d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:44.446089 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524ea3f-07e6-4a2d-8248-6496c2aa162d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:44.446248 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524ea3f-07e6-4a2d-8248-6496c2aa162d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:44.462727 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e524ea3f-07e6-4a2d-8248-6496c2aa162d None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:44.463947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e524ea3f-07e6-4a2d-8248-6496c2aa162d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:44.488632 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:44.531159 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 549/1125] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:44.555590 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd374b-ef1d-4892-8183-b93d1d67973e None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:44.571596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:44.656064 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd374b-ef1d-4892-8183-b93d1d67973e None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:44.689771 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a92e3-675c-4781-9271-f17acfe52af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:44.690569 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a92e3-675c-4781-9271-f17acfe52af9 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:44.690859 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a92e3-675c-4781-9271-f17acfe52af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:44.691174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a92e3-675c-4781-9271-f17acfe52af9 None admin] PATH_INFO: `/v3/roles/61397d4a1ccd4a8ca479f98186ed489b` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:44.693066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a92e3-675c-4781-9271-f17acfe52af9 None admin] RBAC: Authorizing `identity:get_role(role_id=61397d4a1ccd4a8ca479f98186ed489b)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:44.695311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a92e3-675c-4781-9271-f17acfe52af9 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:44.697240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 577/1126] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:24:44 2022] GET /identity/v3/roles/61397d4a1ccd4a8ca479f98186ed489b => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:44.702990 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524ea3f-07e6-4a2d-8248-6496c2aa162d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:44.799597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524ea3f-07e6-4a2d-8248-6496c2aa162d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:44.832301 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7503a-e100-47ef-81ff-776f58cd7486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:44.833200 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7503a-e100-47ef-81ff-776f58cd7486 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:44.833487 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7503a-e100-47ef-81ff-776f58cd7486 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:44.833772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7503a-e100-47ef-81ff-776f58cd7486 None admin] PATH_INFO: `/v3/users/dac15ae97fd34a718327e4c09f36c33c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:44.834980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b7503a-e100-47ef-81ff-776f58cd7486 None admin] RBAC: Authorizing `identity:get_user(user_id=dac15ae97fd34a718327e4c09f36c33c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:44.836469 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b7503a-e100-47ef-81ff-776f58cd7486 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:44.838082 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 550/1127] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:24:44 2022] GET /identity/v3/users/dac15ae97fd34a718327e4c09f36c33c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:44.843768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a92e3-675c-4781-9271-f17acfe52af9 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:44.924620 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:44.942946 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a92e3-675c-4781-9271-f17acfe52af9 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:44.977007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f157a2e-d75a-445f-b9b7-bea77c0d1fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:44.977653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f157a2e-d75a-445f-b9b7-bea77c0d1fee None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:44.977934 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f157a2e-d75a-445f-b9b7-bea77c0d1fee None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:44.978212 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f157a2e-d75a-445f-b9b7-bea77c0d1fee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:45.042156 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f157a2e-d75a-445f-b9b7-bea77c0d1fee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:45.043611 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f157a2e-d75a-445f-b9b7-bea77c0d1fee None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:45.119256 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:45.138192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 578/1128] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:24:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:24:45.143949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7503a-e100-47ef-81ff-776f58cd7486 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:45.175029 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:45.243222 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7503a-e100-47ef-81ff-776f58cd7486 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:45.276298 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a42998-1eb5-4b81-a107-1c51db73c79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:45.277151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a42998-1eb5-4b81-a107-1c51db73c79b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:45.277440 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a42998-1eb5-4b81-a107-1c51db73c79b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:45.277723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a42998-1eb5-4b81-a107-1c51db73c79b None admin] PATH_INFO: `/v3/projects/c4a7b621f2b149eb9e972bb032e21bda` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:45.278893 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a42998-1eb5-4b81-a107-1c51db73c79b None admin] RBAC: Authorizing `identity:get_project(project_id=c4a7b621f2b149eb9e972bb032e21bda)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:45.280360 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a42998-1eb5-4b81-a107-1c51db73c79b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:45.282049 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 551/1129] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:24:45 2022] GET /identity/v3/projects/c4a7b621f2b149eb9e972bb032e21bda => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:45.288698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f157a2e-d75a-445f-b9b7-bea77c0d1fee None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:45.384852 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f157a2e-d75a-445f-b9b7-bea77c0d1fee None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:45.417716 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db765f-8cda-45ce-a63f-e329ea304f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:45.418358 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db765f-8cda-45ce-a63f-e329ea304f3a None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:45.418642 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db765f-8cda-45ce-a63f-e329ea304f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:45.418947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db765f-8cda-45ce-a63f-e329ea304f3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:45.419764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db765f-8cda-45ce-a63f-e329ea304f3a None admin] RBAC: Adding query filter params (role.id=61397d4a1ccd4a8ca479f98186ed489b, scope.project.id=c4a7b621f2b149eb9e972bb032e21bda, user.id=dac15ae97fd34a718327e4c09f36c33c) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:45.420117 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db765f-8cda-45ce-a63f-e329ea304f3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:45.421556 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db765f-8cda-45ce-a63f-e329ea304f3a None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:45.427353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 579/1130] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:24:45 2022] GET /identity/v3/role_assignments?user.id=dac15ae97fd34a718327e4c09f36c33c&scope.project.id=c4a7b621f2b149eb9e972bb032e21bda&role.id=61397d4a1ccd4a8ca479f98186ed489b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:47.487874 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9b3c4-01a3-487b-bfa2-f84f2c7f3da2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:47.488412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9b3c4-01a3-487b-bfa2-f84f2c7f3da2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:47.488412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9b3c4-01a3-487b-bfa2-f84f2c7f3da2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:47.489858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 552/1131] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:47.494983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7987c-fdaf-43e3-b9af-7faf49194974 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:47.495407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7987c-fdaf-43e3-b9af-7faf49194974 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:47.495552 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7987c-fdaf-43e3-b9af-7faf49194974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:47.512016 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-38a7987c-fdaf-43e3-b9af-7faf49194974 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:47.513189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-38a7987c-fdaf-43e3-b9af-7faf49194974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:47.553821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:47.577247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 580/1132] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:47.601603 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9b3c4-01a3-487b-bfa2-f84f2c7f3da2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:47.696501 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:47.701240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9b3c4-01a3-487b-bfa2-f84f2c7f3da2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:47.734113 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ee3e0-b31f-400b-b052-80ef7b691f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:47.734886 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ee3e0-b31f-400b-b052-80ef7b691f30 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:47.735204 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ee3e0-b31f-400b-b052-80ef7b691f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:47.735484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ee3e0-b31f-400b-b052-80ef7b691f30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:47.736638 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ee3e0-b31f-400b-b052-80ef7b691f30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:47.738103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ee3e0-b31f-400b-b052-80ef7b691f30 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:47.739673 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 553/1133] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:47.745338 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7987c-fdaf-43e3-b9af-7faf49194974 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:47.844348 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7987c-fdaf-43e3-b9af-7faf49194974 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:47.877793 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f9508-dcdc-4194-983d-4cffaf83b4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:47.878450 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f9508-dcdc-4194-983d-4cffaf83b4f2 None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:47.878736 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f9508-dcdc-4194-983d-4cffaf83b4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:47.879054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f9508-dcdc-4194-983d-4cffaf83b4f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:47.879750 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f9508-dcdc-4194-983d-4cffaf83b4f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:47.881225 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f9508-dcdc-4194-983d-4cffaf83b4f2 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:47.904919 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 581/1134] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 22:24:47 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:24:50.009200 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016f5de-554a-4dc9-a272-67cc5d2e807b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:50.009759 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016f5de-554a-4dc9-a272-67cc5d2e807b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:50.009759 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016f5de-554a-4dc9-a272-67cc5d2e807b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:50.011180 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 554/1135] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:50.016381 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53165eff-f456-4e02-83cb-dc9044b279b8 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:50.016801 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53165eff-f456-4e02-83cb-dc9044b279b8 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:50.016950 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53165eff-f456-4e02-83cb-dc9044b279b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:50.033543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-53165eff-f456-4e02-83cb-dc9044b279b8 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:50.034711 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-53165eff-f456-4e02-83cb-dc9044b279b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:50.057953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:50.102740 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 582/1136] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:24:50.128028 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016f5de-554a-4dc9-a272-67cc5d2e807b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:50.227393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016f5de-554a-4dc9-a272-67cc5d2e807b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:50.260400 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1a187-ef63-4e14-b7de-d0d1d16a2fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:50.261118 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1a187-ef63-4e14-b7de-d0d1d16a2fee None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:50.261407 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1a187-ef63-4e14-b7de-d0d1d16a2fee None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:50.261689 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1a187-ef63-4e14-b7de-d0d1d16a2fee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:50.262844 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c1a187-ef63-4e14-b7de-d0d1d16a2fee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:50.264300 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c1a187-ef63-4e14-b7de-d0d1d16a2fee None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:50.265887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 555/1137] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:24:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:24:50.272085 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53165eff-f456-4e02-83cb-dc9044b279b8 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:50.373168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53165eff-f456-4e02-83cb-dc9044b279b8 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:50.406281 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441999b6-e2b6-4ac7-a2a9-205de72a96ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:50.406978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441999b6-e2b6-4ac7-a2a9-205de72a96ea None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:50.407271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441999b6-e2b6-4ac7-a2a9-205de72a96ea None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:50.407556 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441999b6-e2b6-4ac7-a2a9-205de72a96ea None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:50.408276 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441999b6-e2b6-4ac7-a2a9-205de72a96ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:50.409710 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441999b6-e2b6-4ac7-a2a9-205de72a96ea None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:50.429581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 583/1138] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 22:24:50 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:24:52.489322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60d3d8-bdf5-4f58-bb6b-68f775f7736a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:52.489981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60d3d8-bdf5-4f58-bb6b-68f775f7736a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:52.489981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60d3d8-bdf5-4f58-bb6b-68f775f7736a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:52.491280 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 556/1139] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:52.496389 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd553206-e941-4b35-8a72-b29ba3504948 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:52.496781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd553206-e941-4b35-8a72-b29ba3504948 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:52.496967 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd553206-e941-4b35-8a72-b29ba3504948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:52.514048 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-cd553206-e941-4b35-8a72-b29ba3504948 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:52.515274 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-cd553206-e941-4b35-8a72-b29ba3504948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:52.525271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:52.582062 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 584/1140] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:52.605454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60d3d8-bdf5-4f58-bb6b-68f775f7736a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:52.640369 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:52.705139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60d3d8-bdf5-4f58-bb6b-68f775f7736a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:52.736816 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44325d4-f7f7-49fd-87cb-0cc7db9f20d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:52.737584 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44325d4-f7f7-49fd-87cb-0cc7db9f20d3 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:52.737871 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44325d4-f7f7-49fd-87cb-0cc7db9f20d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:52.738152 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44325d4-f7f7-49fd-87cb-0cc7db9f20d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:52.748443 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44325d4-f7f7-49fd-87cb-0cc7db9f20d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:24:52.756322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-e44325d4-f7f7-49fd-87cb-0cc7db9f20d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:24:52.756322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:52.756322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:52.756322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:52.756322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:52.756322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:52.756322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:52.756322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:52.756322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:52.756322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:52.756322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:52.756322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:52.756322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:24:52.756322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:24:52.756322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:24:52.756322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:24:52.757434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:24:52.757434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:24:52.757434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:24:52.757434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:24:52.757434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:52.757434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:52.757434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:52.757434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:52.757434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:52.757434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:52.757434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:52.757434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:52.757434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:52.757434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:52.757434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:52.757434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:52.758483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:52.758483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:52.758483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:52.758483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:52.758483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:24:52.758483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:24:52.758483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:24:52.758483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:24:52.758483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:24:52.758483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:24:52.758483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:24:52.758483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:52.758483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 557/1141] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:24:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:52.763392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd553206-e941-4b35-8a72-b29ba3504948 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:52.856608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd553206-e941-4b35-8a72-b29ba3504948 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:52.889435 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72608c1f-7db6-4ef2-9426-c5e00c804cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:52.890088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72608c1f-7db6-4ef2-9426-c5e00c804cde None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:52.890372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72608c1f-7db6-4ef2-9426-c5e00c804cde None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:52.890647 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72608c1f-7db6-4ef2-9426-c5e00c804cde None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:52.891419 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72608c1f-7db6-4ef2-9426-c5e00c804cde None admin] RBAC: Adding query filter params (name=admin) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:52.891728 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72608c1f-7db6-4ef2-9426-c5e00c804cde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:52.893095 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72608c1f-7db6-4ef2-9426-c5e00c804cde None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:52.901152 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 585/1142] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:24:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:52.906761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44325d4-f7f7-49fd-87cb-0cc7db9f20d3 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:52.999530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:53.006768 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44325d4-f7f7-49fd-87cb-0cc7db9f20d3 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:53.040306 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca852ec-3ca8-4f1c-a248-f0d8d73d2a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:53.041173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca852ec-3ca8-4f1c-a248-f0d8d73d2a04 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:53.041461 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca852ec-3ca8-4f1c-a248-f0d8d73d2a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:53.041745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca852ec-3ca8-4f1c-a248-f0d8d73d2a04 None admin] PATH_INFO: `/v3/users/4537969a5e0a46d494e587ca00ef21ce` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:53.055324 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca852ec-3ca8-4f1c-a248-f0d8d73d2a04 None admin] RBAC: Authorizing `identity:get_user(user_id=4537969a5e0a46d494e587ca00ef21ce)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:53.056764 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca852ec-3ca8-4f1c-a248-f0d8d73d2a04 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:53.058425 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 558/1143] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:24:52 2022] GET /identity/v3/users/4537969a5e0a46d494e587ca00ef21ce => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:53.064187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72608c1f-7db6-4ef2-9426-c5e00c804cde None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:53.073210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:53.162901 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72608c1f-7db6-4ef2-9426-c5e00c804cde None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:53.194964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfb55b-ea3c-40c1-a7d8-1e18f76b4189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:53.195620 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfb55b-ea3c-40c1-a7d8-1e18f76b4189 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:53.195911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfb55b-ea3c-40c1-a7d8-1e18f76b4189 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:53.196191 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfb55b-ea3c-40c1-a7d8-1e18f76b4189 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:53.261019 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcfb55b-ea3c-40c1-a7d8-1e18f76b4189 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:53.262407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcfb55b-ea3c-40c1-a7d8-1e18f76b4189 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:53.269688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:53.358359 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 586/1144] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:24:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:24:53.364258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca852ec-3ca8-4f1c-a248-f0d8d73d2a04 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:53.459987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca852ec-3ca8-4f1c-a248-f0d8d73d2a04 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:53.492834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd48ba1-3e55-451b-a0c3-77a896fbe1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:53.493684 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd48ba1-3e55-451b-a0c3-77a896fbe1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:53.493971 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd48ba1-3e55-451b-a0c3-77a896fbe1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:53.494251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd48ba1-3e55-451b-a0c3-77a896fbe1c6 None admin] PATH_INFO: `/v3/projects/13fdd8b23680490eb76a94e1d7e70e93` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:53.495453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd48ba1-3e55-451b-a0c3-77a896fbe1c6 None admin] RBAC: Authorizing `identity:get_project(project_id=13fdd8b23680490eb76a94e1d7e70e93)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:53.496896 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd48ba1-3e55-451b-a0c3-77a896fbe1c6 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:53.498572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 559/1145] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:24:53 2022] GET /identity/v3/projects/13fdd8b23680490eb76a94e1d7e70e93 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:53.504474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfb55b-ea3c-40c1-a7d8-1e18f76b4189 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:53.600636 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfb55b-ea3c-40c1-a7d8-1e18f76b4189 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:53.608290 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:53.635227 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed959473-81fa-4248-9365-8c4541d8127d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:53.635882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed959473-81fa-4248-9365-8c4541d8127d None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:53.636170 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed959473-81fa-4248-9365-8c4541d8127d None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:53.636451 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed959473-81fa-4248-9365-8c4541d8127d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:53.637282 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed959473-81fa-4248-9365-8c4541d8127d None admin] RBAC: Adding query filter params (role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.project.id=13fdd8b23680490eb76a94e1d7e70e93, user.id=4537969a5e0a46d494e587ca00ef21ce) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:53.637603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed959473-81fa-4248-9365-8c4541d8127d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:53.639056 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed959473-81fa-4248-9365-8c4541d8127d None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:53.644353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 587/1146] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:24:53 2022] GET /identity/v3/role_assignments?user.id=4537969a5e0a46d494e587ca00ef21ce&scope.project.id=13fdd8b23680490eb76a94e1d7e70e93&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:55.652791 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a5d2e-68e9-426a-b989-a362268eba1f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:55.653316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a5d2e-68e9-426a-b989-a362268eba1f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:55.653430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88a5d2e-68e9-426a-b989-a362268eba1f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:55.654727 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 560/1147] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:55.659974 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d25f1-7664-4ed5-b982-125207633ec2 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:55.660371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d25f1-7664-4ed5-b982-125207633ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:55.660546 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d25f1-7664-4ed5-b982-125207633ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:55.677771 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-f00d25f1-7664-4ed5-b982-125207633ec2 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:55.678991 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-f00d25f1-7664-4ed5-b982-125207633ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:55.743406 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 588/1148] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:55.765385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a5d2e-68e9-426a-b989-a362268eba1f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:55.787764 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:55.865415 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88a5d2e-68e9-426a-b989-a362268eba1f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:55.898142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918254f-f5c0-4aad-b4d7-66c2e1858bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:55.899194 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918254f-f5c0-4aad-b4d7-66c2e1858bfd None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:55.899481 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918254f-f5c0-4aad-b4d7-66c2e1858bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:55.899761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918254f-f5c0-4aad-b4d7-66c2e1858bfd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:55.909456 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918254f-f5c0-4aad-b4d7-66c2e1858bfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:24:55.917495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a918254f-f5c0-4aad-b4d7-66c2e1858bfd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:24:55.917495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:55.917495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:55.917495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:55.917495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:55.917495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:55.917495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:55.917495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:55.917495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:55.917495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:55.917495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:55.917495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:55.917495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:24:55.917495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:24:55.917495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:24:55.917495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:24:55.918457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:24:55.918457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:24:55.918457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:24:55.918457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:24:55.918457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:55.918457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:55.918457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:55.918457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:55.918457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:55.918457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:55.918457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:55.918457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:55.918457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:55.918457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:55.918457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:55.918457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:55.919390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:55.919390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:55.919390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:55.919390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:55.919390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:24:55.919390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:24:55.919390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:24:55.919390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:24:55.919390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:24:55.919390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:24:55.919390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:24:55.919390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:24:55.919390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 561/1149] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:24:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:55.924739 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d25f1-7664-4ed5-b982-125207633ec2 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:56.017412 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:56.021225 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d25f1-7664-4ed5-b982-125207633ec2 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:56.054891 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b4243-2f42-48c6-9e8e-3fe63b23c1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:56.055595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b4243-2f42-48c6-9e8e-3fe63b23c1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:56.055885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b4243-2f42-48c6-9e8e-3fe63b23c1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:56.056169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b4243-2f42-48c6-9e8e-3fe63b23c1a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:56.056918 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b4243-2f42-48c6-9e8e-3fe63b23c1a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:56.057233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b4243-2f42-48c6-9e8e-3fe63b23c1a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:56.058619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b4243-2f42-48c6-9e8e-3fe63b23c1a3 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:56.066840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 589/1150] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:24:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:56.073036 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918254f-f5c0-4aad-b4d7-66c2e1858bfd None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:56.153564 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:56.172350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918254f-f5c0-4aad-b4d7-66c2e1858bfd None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:56.204055 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7a890-d695-4fea-9053-e18d93da64f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:56.204913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7a890-d695-4fea-9053-e18d93da64f0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:56.205197 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7a890-d695-4fea-9053-e18d93da64f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:56.205476 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7a890-d695-4fea-9053-e18d93da64f0 None admin] PATH_INFO: `/v3/users/4537969a5e0a46d494e587ca00ef21ce` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:56.206636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7a890-d695-4fea-9053-e18d93da64f0 None admin] RBAC: Authorizing `identity:get_user(user_id=4537969a5e0a46d494e587ca00ef21ce)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:56.208668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f7a890-d695-4fea-9053-e18d93da64f0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:56.210403 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 562/1151] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:24:56 2022] GET /identity/v3/users/4537969a5e0a46d494e587ca00ef21ce => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:56.216230 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b4243-2f42-48c6-9e8e-3fe63b23c1a3 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:56.312326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b4243-2f42-48c6-9e8e-3fe63b23c1a3 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:56.346140 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929cebe-1f92-4ff9-8598-4513f4223fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:56.347471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929cebe-1f92-4ff9-8598-4513f4223fae None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:56.347765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929cebe-1f92-4ff9-8598-4513f4223fae None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:56.348048 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929cebe-1f92-4ff9-8598-4513f4223fae None admin] PATH_INFO: `/v3/projects/13fdd8b23680490eb76a94e1d7e70e93` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:56.349355 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929cebe-1f92-4ff9-8598-4513f4223fae None admin] RBAC: Authorizing `identity:get_project(project_id=13fdd8b23680490eb76a94e1d7e70e93)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:56.351831 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929cebe-1f92-4ff9-8598-4513f4223fae None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:56.353539 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 590/1152] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:24:56 2022] GET /identity/v3/projects/13fdd8b23680490eb76a94e1d7e70e93 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:56.359435 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7a890-d695-4fea-9053-e18d93da64f0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:56.455383 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7a890-d695-4fea-9053-e18d93da64f0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:56.486676 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a61f0-4d7a-45ff-8ebe-2a3f3a814a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:56.487570 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a61f0-4d7a-45ff-8ebe-2a3f3a814a0c None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:56.487857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a61f0-4d7a-45ff-8ebe-2a3f3a814a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:56.488137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a61f0-4d7a-45ff-8ebe-2a3f3a814a0c None admin] PATH_INFO: `/v3/projects/13fdd8b23680490eb76a94e1d7e70e93/users/4537969a5e0a46d494e587ca00ef21ce/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:56.490696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a61f0-4d7a-45ff-8ebe-2a3f3a814a0c None admin] RBAC: Authorizing `identity:create_grant(project_id=13fdd8b23680490eb76a94e1d7e70e93, user_id=4537969a5e0a46d494e587ca00ef21ce, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:56.492128 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a61f0-4d7a-45ff-8ebe-2a3f3a814a0c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:56.504590 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 563/1153] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:24:56 2022] PUT /identity/v3/projects/13fdd8b23680490eb76a94e1d7e70e93/users/4537969a5e0a46d494e587ca00ef21ce/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:24:58.657937 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c136fa-5a58-4f50-8fc0-2090a6ea7843 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:58.658538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c136fa-5a58-4f50-8fc0-2090a6ea7843 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:58.658634 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c136fa-5a58-4f50-8fc0-2090a6ea7843 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:58.660401 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 591/1154] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:24:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:24:58.666885 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54436367-aefd-4565-beac-fdb6a64259b5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:58.667354 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54436367-aefd-4565-beac-fdb6a64259b5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:58.667503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54436367-aefd-4565-beac-fdb6a64259b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:58.684080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-54436367-aefd-4565-beac-fdb6a64259b5 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:24:58.685314 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-54436367-aefd-4565-beac-fdb6a64259b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:24:58.687336 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:58.754273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 564/1155] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:24:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:24:58.779125 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c136fa-5a58-4f50-8fc0-2090a6ea7843 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:58.810490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:58.882976 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c136fa-5a58-4f50-8fc0-2090a6ea7843 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:58.916889 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a928c-1a97-4bf8-a3a3-e8e2f2bc6a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:58.917634 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a928c-1a97-4bf8-a3a3-e8e2f2bc6a73 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:58.917923 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a928c-1a97-4bf8-a3a3-e8e2f2bc6a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:58.918203 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a928c-1a97-4bf8-a3a3-e8e2f2bc6a73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:58.929072 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a928c-1a97-4bf8-a3a3-e8e2f2bc6a73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:24:58.936574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-6e3a928c-1a97-4bf8-a3a3-e8e2f2bc6a73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:24:58.936574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:24:58.936574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:24:58.936574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:24:58.936574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:24:58.936574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:24:58.936574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:24:58.936574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:24:58.936574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:24:58.936574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:24:58.936574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:24:58.936574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:24:58.936574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:24:58.936574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:24:58.936574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:24:58.936574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:24:58.937698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:24:58.937698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:24:58.937698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:24:58.937698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:24:58.937698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:24:58.937698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:24:58.937698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:24:58.937698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:24:58.937698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:24:58.937698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:24:58.937698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:24:58.937698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:24:58.937698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:24:58.937698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:24:58.937698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:24:58.937698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:24:58.938879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:24:58.938879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:24:58.938879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:24:58.938879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:24:58.938879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:24:58.938879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:24:58.938879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:24:58.938879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:24:58.938879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:24:58.938879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:24:58.938879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:24:58.938879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:24:58.938879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 592/1156] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:24:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:24:58.944961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54436367-aefd-4565-beac-fdb6a64259b5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:59.054861 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54436367-aefd-4565-beac-fdb6a64259b5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:59.092916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e6b6a-3284-44fc-8d32-0d07a8bfe97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:59.093671 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e6b6a-3284-44fc-8d32-0d07a8bfe97e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:59.094029 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e6b6a-3284-44fc-8d32-0d07a8bfe97e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:59.094368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e6b6a-3284-44fc-8d32-0d07a8bfe97e None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:59.095248 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e6b6a-3284-44fc-8d32-0d07a8bfe97e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:59.095632 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e6b6a-3284-44fc-8d32-0d07a8bfe97e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:59.097236 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e6b6a-3284-44fc-8d32-0d07a8bfe97e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:59.103439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:59.107972 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 565/1157] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:24:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:59.113512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a928c-1a97-4bf8-a3a3-e8e2f2bc6a73 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:59.203440 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:59.213940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a928c-1a97-4bf8-a3a3-e8e2f2bc6a73 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:59.247384 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf6224-fabd-47f9-8d84-3ed106ff5d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:59.248259 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf6224-fabd-47f9-8d84-3ed106ff5d6e None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:59.248548 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf6224-fabd-47f9-8d84-3ed106ff5d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:59.248832 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf6224-fabd-47f9-8d84-3ed106ff5d6e None admin] PATH_INFO: `/v3/users/4537969a5e0a46d494e587ca00ef21ce` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:59.250029 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf6224-fabd-47f9-8d84-3ed106ff5d6e None admin] RBAC: Authorizing `identity:get_user(user_id=4537969a5e0a46d494e587ca00ef21ce)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:59.251510 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf6224-fabd-47f9-8d84-3ed106ff5d6e None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:59.253128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 593/1158] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:24:59 2022] GET /identity/v3/users/4537969a5e0a46d494e587ca00ef21ce => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:59.260170 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e6b6a-3284-44fc-8d32-0d07a8bfe97e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:59.361495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e6b6a-3284-44fc-8d32-0d07a8bfe97e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:59.398291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045fe4c-766d-429d-8d66-ffabc1040889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:59.399226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045fe4c-766d-429d-8d66-ffabc1040889 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:59.399517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045fe4c-766d-429d-8d66-ffabc1040889 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:59.399801 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045fe4c-766d-429d-8d66-ffabc1040889 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:59.464649 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:59.466915 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045fe4c-766d-429d-8d66-ffabc1040889 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:59.468924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045fe4c-766d-429d-8d66-ffabc1040889 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:59.567408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 566/1159] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:24:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 22:24:59.573714 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf6224-fabd-47f9-8d84-3ed106ff5d6e None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:59.673827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf6224-fabd-47f9-8d84-3ed106ff5d6e None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:59.706788 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6752a47-2ec0-4fd4-a7c6-7cf22c37aa3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:59.707680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6752a47-2ec0-4fd4-a7c6-7cf22c37aa3d None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:59.708036 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6752a47-2ec0-4fd4-a7c6-7cf22c37aa3d None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:59.708825 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6752a47-2ec0-4fd4-a7c6-7cf22c37aa3d None admin] PATH_INFO: `/v3/projects/13fdd8b23680490eb76a94e1d7e70e93` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:59.710070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6752a47-2ec0-4fd4-a7c6-7cf22c37aa3d None admin] RBAC: Authorizing `identity:get_project(project_id=13fdd8b23680490eb76a94e1d7e70e93)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:59.711560 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6752a47-2ec0-4fd4-a7c6-7cf22c37aa3d None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:59.713227 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 594/1160] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:24:59 2022] GET /identity/v3/projects/13fdd8b23680490eb76a94e1d7e70e93 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:24:59.719398 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045fe4c-766d-429d-8d66-ffabc1040889 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:24:59.818211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:24:59.821264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045fe4c-766d-429d-8d66-ffabc1040889 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:24:59.856222 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cc84e-9c74-48bf-8d3a-8cb8ea8c69a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:24:59.856871 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cc84e-9c74-48bf-8d3a-8cb8ea8c69a0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:24:59.857159 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cc84e-9c74-48bf-8d3a-8cb8ea8c69a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:24:59.857446 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cc84e-9c74-48bf-8d3a-8cb8ea8c69a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:24:59.858267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cc84e-9c74-48bf-8d3a-8cb8ea8c69a0 None admin] RBAC: Adding query filter params (role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.project.id=13fdd8b23680490eb76a94e1d7e70e93, user.id=4537969a5e0a46d494e587ca00ef21ce) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:24:59.858584 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cc84e-9c74-48bf-8d3a-8cb8ea8c69a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:24:59.860042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cc84e-9c74-48bf-8d3a-8cb8ea8c69a0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:24:59.865996 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 567/1161] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:24:59 2022] GET /identity/v3/role_assignments?user.id=4537969a5e0a46d494e587ca00ef21ce&scope.project.id=13fdd8b23680490eb76a94e1d7e70e93&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:25:01.931577 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7d939-30e6-42c2-896f-5f9253cffef8 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:01.932212 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7d939-30e6-42c2-896f-5f9253cffef8 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:01.932212 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7d939-30e6-42c2-896f-5f9253cffef8 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:01.933494 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 595/1162] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:25:01.938735 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b2b30-166c-48a3-889a-e4bf60408c18 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:01.939195 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b2b30-166c-48a3-889a-e4bf60408c18 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:01.939345 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b2b30-166c-48a3-889a-e4bf60408c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:01.966543 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9a4b2b30-166c-48a3-889a-e4bf60408c18 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:25:01.967769 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9a4b2b30-166c-48a3-889a-e4bf60408c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:25:02.039690 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 568/1163] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:25:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:25:02.062492 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7d939-30e6-42c2-896f-5f9253cffef8 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:02.091602 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:02.181774 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7d939-30e6-42c2-896f-5f9253cffef8 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:02.220774 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424aa6eb-36a0-407c-ba9d-67bc96d781f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:02.221517 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424aa6eb-36a0-407c-ba9d-67bc96d781f6 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:02.221850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424aa6eb-36a0-407c-ba9d-67bc96d781f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:02.222213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424aa6eb-36a0-407c-ba9d-67bc96d781f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:02.303037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424aa6eb-36a0-407c-ba9d-67bc96d781f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:02.304718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424aa6eb-36a0-407c-ba9d-67bc96d781f6 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:02.333485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:02.419444 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 596/1164] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:25:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:25:02.425330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b2b30-166c-48a3-889a-e4bf60408c18 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:02.514806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:02.522348 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b2b30-166c-48a3-889a-e4bf60408c18 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:02.556279 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f364b1c-dcbf-4a11-b9f9-20c708f01aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:02.557078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f364b1c-dcbf-4a11-b9f9-20c708f01aef None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:02.557372 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f364b1c-dcbf-4a11-b9f9-20c708f01aef None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:02.557653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f364b1c-dcbf-4a11-b9f9-20c708f01aef None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:02.564180 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f364b1c-dcbf-4a11-b9f9-20c708f01aef None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:02.565617 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f364b1c-dcbf-4a11-b9f9-20c708f01aef None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:02.579582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-3f364b1c-dcbf-4a11-b9f9-20c708f01aef None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 22:25:02.579582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:25:02.579582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 23 22:25:02.579582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 23 22:25:02.579582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:25:02.579582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:25:02.579582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:25:02.579582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:25:02.579582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 23 22:25:02.579582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:25:02.579582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:25:02.579582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:25:02.579582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:25:02.579582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:25:02.579582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:25:02.579582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:25:02.579582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:25:02.580847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:25:02.580847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:25:02.580847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:25:02.580847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:25:02.580847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:25:02.580847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:25:02.580847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:25:02.580847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:25:02.580847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:25:02.580847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:25:02.580847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:25:02.580847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:25:02.580847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:25:02.580847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:25:02.580847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:25:02.580847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:25:02.581933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:25:02.581933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:25:02.581933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:25:02.581933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:25:02.581933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:25:02.581933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:25:02.581933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:25:02.581933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:25:02.581933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:25:02.581933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:25:02.581933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 23 22:25:02.581933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:25:02.581933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 22:25:02.581933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:25:02.581933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 569/1165] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 22:25:02 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:25:02.587326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424aa6eb-36a0-407c-ba9d-67bc96d781f6 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:02.685906 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424aa6eb-36a0-407c-ba9d-67bc96d781f6 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:02.712357 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:02.721023 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd52f9-05ac-4269-bee6-1bd56981aa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:02.721687 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd52f9-05ac-4269-bee6-1bd56981aa15 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:02.721975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd52f9-05ac-4269-bee6-1bd56981aa15 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:02.722258 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd52f9-05ac-4269-bee6-1bd56981aa15 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:02.723005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd52f9-05ac-4269-bee6-1bd56981aa15 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:25:02.723320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd52f9-05ac-4269-bee6-1bd56981aa15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:02.725226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd52f9-05ac-4269-bee6-1bd56981aa15 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:02.732265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 597/1166] 10.222.0.43 () {56 vars in 1115 bytes} [Thu Jun 23 22:25:02 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:25:04.814462 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa76e5-db26-493d-81ee-e126f2528c04 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:04.815053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa76e5-db26-493d-81ee-e126f2528c04 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:04.815053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa76e5-db26-493d-81ee-e126f2528c04 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:04.816439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 570/1167] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:25:04.822302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce40562-47cb-4927-a130-52c70684ee4d None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:04.822721 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce40562-47cb-4927-a130-52c70684ee4d None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:04.822876 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce40562-47cb-4927-a130-52c70684ee4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:04.839394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-4ce40562-47cb-4927-a130-52c70684ee4d None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:25:04.840559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-4ce40562-47cb-4927-a130-52c70684ee4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:25:04.904169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 598/1168] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:25:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:25:04.928079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa76e5-db26-493d-81ee-e126f2528c04 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:05.026462 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa76e5-db26-493d-81ee-e126f2528c04 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:05.059549 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb8ac2-045c-4340-b8cb-6ef70d85dec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:05.060192 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb8ac2-045c-4340-b8cb-6ef70d85dec9 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:05.060478 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb8ac2-045c-4340-b8cb-6ef70d85dec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:05.060758 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb8ac2-045c-4340-b8cb-6ef70d85dec9 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:05.061483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb8ac2-045c-4340-b8cb-6ef70d85dec9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:05.062827 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb8ac2-045c-4340-b8cb-6ef70d85dec9 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:05.076794 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 571/1169] 10.222.0.43 () {60 vars in 1138 bytes} [Thu Jun 23 22:25:04 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:25:07.094782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7986b96-bbe4-45bf-afe5-8cc26f57127f None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:07.095381 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7986b96-bbe4-45bf-afe5-8cc26f57127f None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:07.095381 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7986b96-bbe4-45bf-afe5-8cc26f57127f None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:07.096741 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 599/1170] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:25:07.102384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb638d47-1b96-4511-adb3-60fcae623d96 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:07.102805 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb638d47-1b96-4511-adb3-60fcae623d96 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:07.102986 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb638d47-1b96-4511-adb3-60fcae623d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:07.120275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cb638d47-1b96-4511-adb3-60fcae623d96 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:25:07.121464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cb638d47-1b96-4511-adb3-60fcae623d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:25:07.132922 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:07.190275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 572/1171] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:25:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:25:07.213727 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7986b96-bbe4-45bf-afe5-8cc26f57127f None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:07.313873 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7986b96-bbe4-45bf-afe5-8cc26f57127f None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:07.334074 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:07.347993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4b7ab-2f7d-4abc-a646-6e073cfb3989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:07.349332 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4b7ab-2f7d-4abc-a646-6e073cfb3989 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:07.349621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4b7ab-2f7d-4abc-a646-6e073cfb3989 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:07.349901 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4b7ab-2f7d-4abc-a646-6e073cfb3989 None admin] PATH_INFO: `/v3/domains/b3e2550ea5574b4ca0827b7c2f135045` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:07.360119 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4b7ab-2f7d-4abc-a646-6e073cfb3989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b3e2550ea5574b4ca0827b7c2f135045)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:07.361568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4b7ab-2f7d-4abc-a646-6e073cfb3989 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:07.363215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 600/1172] 10.222.0.43 () {56 vars in 1183 bytes} [Thu Jun 23 22:25:07 2022] GET /identity/v3/domains/b3e2550ea5574b4ca0827b7c2f135045 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:25:07.369100 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb638d47-1b96-4511-adb3-60fcae623d96 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:07.465023 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb638d47-1b96-4511-adb3-60fcae623d96 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:07.498834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4240f502-9464-44f4-b16d-cc93cb3d0242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:07.499513 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240f502-9464-44f4-b16d-cc93cb3d0242 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:07.499802 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240f502-9464-44f4-b16d-cc93cb3d0242 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:07.500087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240f502-9464-44f4-b16d-cc93cb3d0242 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:07.500797 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4240f502-9464-44f4-b16d-cc93cb3d0242 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:07.502213 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4240f502-9464-44f4-b16d-cc93cb3d0242 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:07.515705 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:07.528144 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 573/1173] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 22:25:07 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:25:09.619202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cca0f5-05e6-4d69-9edd-46393b125d1f None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:09.619725 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cca0f5-05e6-4d69-9edd-46393b125d1f None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:09.619835 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cca0f5-05e6-4d69-9edd-46393b125d1f None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:09.621120 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 601/1174] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:25:09.626220 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24b810-1d4b-4391-beeb-87a4b4aa12ca None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:09.626590 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24b810-1d4b-4391-beeb-87a4b4aa12ca None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:09.626787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24b810-1d4b-4391-beeb-87a4b4aa12ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:09.643722 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1f24b810-1d4b-4391-beeb-87a4b4aa12ca None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:25:09.644906 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1f24b810-1d4b-4391-beeb-87a4b4aa12ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:25:09.715024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 574/1175] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:25:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:25:09.739224 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cca0f5-05e6-4d69-9edd-46393b125d1f None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:09.837416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cca0f5-05e6-4d69-9edd-46393b125d1f None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:09.869743 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd46d6-3b2b-4f97-b051-c914f1dfa5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:09.870546 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd46d6-3b2b-4f97-b051-c914f1dfa5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:09.870831 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd46d6-3b2b-4f97-b051-c914f1dfa5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:09.871140 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd46d6-3b2b-4f97-b051-c914f1dfa5d8 None admin] PATH_INFO: `/v3/domains/b3e2550ea5574b4ca0827b7c2f135045` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:09.872294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd46d6-3b2b-4f97-b051-c914f1dfa5d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b3e2550ea5574b4ca0827b7c2f135045)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:09.873721 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd46d6-3b2b-4f97-b051-c914f1dfa5d8 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:09.875329 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 602/1176] 10.222.0.43 () {56 vars in 1183 bytes} [Thu Jun 23 22:25:09 2022] GET /identity/v3/domains/b3e2550ea5574b4ca0827b7c2f135045 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:25:09.881600 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24b810-1d4b-4391-beeb-87a4b4aa12ca None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:09.976411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24b810-1d4b-4391-beeb-87a4b4aa12ca None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:10.000659 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:10.010088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a394df5-a4da-4d1b-8813-4c0aada65788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:10.010757 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a394df5-a4da-4d1b-8813-4c0aada65788 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:10.011081 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a394df5-a4da-4d1b-8813-4c0aada65788 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:10.011368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a394df5-a4da-4d1b-8813-4c0aada65788 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:10.012073 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a394df5-a4da-4d1b-8813-4c0aada65788 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:10.013497 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a394df5-a4da-4d1b-8813-4c0aada65788 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:10.032739 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 575/1177] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 22:25:09 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:25:12.073727 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99186efc-9c90-40de-8d85-e212af94c5e3 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:12.074281 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99186efc-9c90-40de-8d85-e212af94c5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:12.074281 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99186efc-9c90-40de-8d85-e212af94c5e3 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:12.075718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 603/1178] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:25:12.080705 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ed2cb-8202-42cd-a6bb-181508c4e01a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:12.081082 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ed2cb-8202-42cd-a6bb-181508c4e01a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:12.081275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ed2cb-8202-42cd-a6bb-181508c4e01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:12.097704 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bf0ed2cb-8202-42cd-a6bb-181508c4e01a None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:25:12.098914 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bf0ed2cb-8202-42cd-a6bb-181508c4e01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:25:12.167126 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 576/1179] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:25:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:25:12.191171 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99186efc-9c90-40de-8d85-e212af94c5e3 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:12.239955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:12.296051 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99186efc-9c90-40de-8d85-e212af94c5e3 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:12.329519 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c422ed-011d-4561-bd11-e2d0a04fbe23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:12.330236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c422ed-011d-4561-bd11-e2d0a04fbe23 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:12.330521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c422ed-011d-4561-bd11-e2d0a04fbe23 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:12.330800 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c422ed-011d-4561-bd11-e2d0a04fbe23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:12.341122 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c422ed-011d-4561-bd11-e2d0a04fbe23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:25:12.348681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-a7c422ed-011d-4561-bd11-e2d0a04fbe23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:25:12.348681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:25:12.348681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:25:12.348681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:25:12.348681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:25:12.348681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:25:12.348681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:25:12.348681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:25:12.348681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:25:12.348681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:25:12.348681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:25:12.348681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:25:12.348681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:25:12.348681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:25:12.348681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:25:12.348681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:25:12.349782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:25:12.349782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:25:12.349782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:25:12.349782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:25:12.349782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:25:12.349782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:25:12.349782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:25:12.349782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:25:12.349782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:25:12.349782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:25:12.349782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:25:12.349782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:25:12.349782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:25:12.349782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:25:12.349782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:25:12.349782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:25:12.350997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:25:12.350997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:25:12.350997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:25:12.350997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:25:12.350997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:25:12.350997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:25:12.350997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:25:12.350997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:25:12.350997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:25:12.350997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:25:12.350997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:25:12.350997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:25:12.350997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 604/1180] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:25:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:25:12.356009 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ed2cb-8202-42cd-a6bb-181508c4e01a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:12.450645 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ed2cb-8202-42cd-a6bb-181508c4e01a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:12.463042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:12.484254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9564d2c-ea0a-42f0-9eec-7f434f8462d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:12.484907 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9564d2c-ea0a-42f0-9eec-7f434f8462d5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:12.485193 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9564d2c-ea0a-42f0-9eec-7f434f8462d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:12.485470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9564d2c-ea0a-42f0-9eec-7f434f8462d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:12.486206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9564d2c-ea0a-42f0-9eec-7f434f8462d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:25:12.486513 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9564d2c-ea0a-42f0-9eec-7f434f8462d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:12.487896 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9564d2c-ea0a-42f0-9eec-7f434f8462d5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:12.496285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 577/1181] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:25:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:25:12.501896 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c422ed-011d-4561-bd11-e2d0a04fbe23 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:12.598315 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c422ed-011d-4561-bd11-e2d0a04fbe23 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:12.607623 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:12.632188 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7ab0b-a426-49cb-b3a3-98fc5907cd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:12.633054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7ab0b-a426-49cb-b3a3-98fc5907cd7d None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:12.633339 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7ab0b-a426-49cb-b3a3-98fc5907cd7d None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:12.633620 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7ab0b-a426-49cb-b3a3-98fc5907cd7d None admin] PATH_INFO: `/v3/users/5f0af4c16db14e1f85719d26eb1aab68` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:12.646258 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7ab0b-a426-49cb-b3a3-98fc5907cd7d None admin] RBAC: Authorizing `identity:get_user(user_id=5f0af4c16db14e1f85719d26eb1aab68)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:12.647717 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7ab0b-a426-49cb-b3a3-98fc5907cd7d None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:12.649588 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 605/1182] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:25:12 2022] GET /identity/v3/users/5f0af4c16db14e1f85719d26eb1aab68 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:25:12.655359 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9564d2c-ea0a-42f0-9eec-7f434f8462d5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:12.751728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9564d2c-ea0a-42f0-9eec-7f434f8462d5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:12.783407 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8cec74-d430-47d5-ad2d-fc591f21f324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:12.784042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cec74-d430-47d5-ad2d-fc591f21f324 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:12.784329 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cec74-d430-47d5-ad2d-fc591f21f324 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:12.784634 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cec74-d430-47d5-ad2d-fc591f21f324 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:12.825799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:12.854320 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8cec74-d430-47d5-ad2d-fc591f21f324 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:12.855743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8cec74-d430-47d5-ad2d-fc591f21f324 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:12.952060 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 578/1183] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:25:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:25:12.958442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7ab0b-a426-49cb-b3a3-98fc5907cd7d None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:13.054758 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7ab0b-a426-49cb-b3a3-98fc5907cd7d None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:13.088430 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd7fd6-251a-46ff-8f0e-aa1b04207432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:13.089290 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd7fd6-251a-46ff-8f0e-aa1b04207432 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:13.089579 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd7fd6-251a-46ff-8f0e-aa1b04207432 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:13.089860 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd7fd6-251a-46ff-8f0e-aa1b04207432 None admin] PATH_INFO: `/v3/projects/a3cabc6461bf4f29bb9c8973bdade54e` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:13.091062 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd7fd6-251a-46ff-8f0e-aa1b04207432 None admin] RBAC: Authorizing `identity:get_project(project_id=a3cabc6461bf4f29bb9c8973bdade54e)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:13.092498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd7fd6-251a-46ff-8f0e-aa1b04207432 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:13.094165 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 606/1184] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:25:12 2022] GET /identity/v3/projects/a3cabc6461bf4f29bb9c8973bdade54e => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:25:13.100649 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8cec74-d430-47d5-ad2d-fc591f21f324 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:13.173741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:13.198338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8cec74-d430-47d5-ad2d-fc591f21f324 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:13.230525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b2e12-8b84-4b94-9200-feec5efb6284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:13.231191 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b2e12-8b84-4b94-9200-feec5efb6284 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:13.231478 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b2e12-8b84-4b94-9200-feec5efb6284 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:13.231757 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b2e12-8b84-4b94-9200-feec5efb6284 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:13.232574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b2e12-8b84-4b94-9200-feec5efb6284 None admin] RBAC: Adding query filter params (role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.project.id=a3cabc6461bf4f29bb9c8973bdade54e, user.id=5f0af4c16db14e1f85719d26eb1aab68) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:25:13.232883 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b2e12-8b84-4b94-9200-feec5efb6284 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:13.234290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b2e12-8b84-4b94-9200-feec5efb6284 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:13.239839 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 579/1185] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:25:13 2022] GET /identity/v3/role_assignments?user.id=5f0af4c16db14e1f85719d26eb1aab68&scope.project.id=a3cabc6461bf4f29bb9c8973bdade54e&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:25:15.220344 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c047449-eeee-42fa-b866-a84b38aadd84 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:15.220872 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c047449-eeee-42fa-b866-a84b38aadd84 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:15.220976 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c047449-eeee-42fa-b866-a84b38aadd84 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:15.222271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 607/1186] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:25:15.227739 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0479fd2-90db-4026-b827-b2da1c92bfa2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:15.228177 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0479fd2-90db-4026-b827-b2da1c92bfa2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:15.228314 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0479fd2-90db-4026-b827-b2da1c92bfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:15.245216 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f0479fd2-90db-4026-b827-b2da1c92bfa2 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:25:15.246405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f0479fd2-90db-4026-b827-b2da1c92bfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:25:15.311521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 580/1187] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:25:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:25:15.334137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c047449-eeee-42fa-b866-a84b38aadd84 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:15.369138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:15.434539 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c047449-eeee-42fa-b866-a84b38aadd84 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:15.468562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9459ab5-fc62-4435-8396-eb0be09d949f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:15.469344 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9459ab5-fc62-4435-8396-eb0be09d949f None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:15.469630 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9459ab5-fc62-4435-8396-eb0be09d949f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:15.469912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9459ab5-fc62-4435-8396-eb0be09d949f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:15.479946 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9459ab5-fc62-4435-8396-eb0be09d949f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:25:15.487456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-b9459ab5-fc62-4435-8396-eb0be09d949f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:25:15.487456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:25:15.487456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:25:15.487456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:25:15.487456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:25:15.487456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:25:15.487456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:25:15.487456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:25:15.487456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:25:15.487456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:25:15.487456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:25:15.487456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:25:15.487456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:25:15.487456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:25:15.487456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:25:15.487456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:25:15.488532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:25:15.488532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:25:15.488532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:25:15.488532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:25:15.488532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:25:15.488532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:25:15.488532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:25:15.488532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:25:15.488532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:25:15.488532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:25:15.488532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:25:15.488532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:25:15.488532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:25:15.488532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:25:15.488532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:25:15.488532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:25:15.489528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:25:15.489528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:25:15.489528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:25:15.489528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:25:15.489528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:25:15.489528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:25:15.489528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:25:15.489528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:25:15.489528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:25:15.489528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:25:15.489528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:25:15.489528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:25:15.489528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 608/1188] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:25:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:25:15.494864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0479fd2-90db-4026-b827-b2da1c92bfa2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:15.559129 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:15.592516 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0479fd2-90db-4026-b827-b2da1c92bfa2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:15.625850 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a21d4-35ba-41f8-969d-72b43f68d939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:15.626530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a21d4-35ba-41f8-969d-72b43f68d939 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:15.626820 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a21d4-35ba-41f8-969d-72b43f68d939 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:15.627132 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a21d4-35ba-41f8-969d-72b43f68d939 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:15.627876 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a21d4-35ba-41f8-969d-72b43f68d939 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:25:15.628189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a21d4-35ba-41f8-969d-72b43f68d939 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:15.629548 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a21d4-35ba-41f8-969d-72b43f68d939 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:15.637518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 581/1189] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:25:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:25:15.643282 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9459ab5-fc62-4435-8396-eb0be09d949f None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:15.732850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:15.739908 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9459ab5-fc62-4435-8396-eb0be09d949f None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:15.771582 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732e154-0b48-482a-b321-58c89f6c5dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:15.772433 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732e154-0b48-482a-b321-58c89f6c5dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:15.772713 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732e154-0b48-482a-b321-58c89f6c5dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:15.772987 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732e154-0b48-482a-b321-58c89f6c5dd8 None admin] PATH_INFO: `/v3/users/5f0af4c16db14e1f85719d26eb1aab68` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:15.774400 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e732e154-0b48-482a-b321-58c89f6c5dd8 None admin] RBAC: Authorizing `identity:get_user(user_id=5f0af4c16db14e1f85719d26eb1aab68)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:15.775846 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e732e154-0b48-482a-b321-58c89f6c5dd8 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:15.777419 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 609/1190] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:25:15 2022] GET /identity/v3/users/5f0af4c16db14e1f85719d26eb1aab68 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:25:15.783092 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a21d4-35ba-41f8-969d-72b43f68d939 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:15.881349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a21d4-35ba-41f8-969d-72b43f68d939 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:15.908706 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:15.915352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d452aec-f1f0-4828-ad11-c794d4ed656b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:15.916198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d452aec-f1f0-4828-ad11-c794d4ed656b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:15.916490 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d452aec-f1f0-4828-ad11-c794d4ed656b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:15.916773 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d452aec-f1f0-4828-ad11-c794d4ed656b None admin] PATH_INFO: `/v3/projects/a3cabc6461bf4f29bb9c8973bdade54e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:15.917950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d452aec-f1f0-4828-ad11-c794d4ed656b None admin] RBAC: Authorizing `identity:get_project(project_id=a3cabc6461bf4f29bb9c8973bdade54e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:15.919416 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d452aec-f1f0-4828-ad11-c794d4ed656b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:15.921099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 582/1191] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:25:15 2022] GET /identity/v3/projects/a3cabc6461bf4f29bb9c8973bdade54e => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:25:15.926909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732e154-0b48-482a-b321-58c89f6c5dd8 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:16.022577 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732e154-0b48-482a-b321-58c89f6c5dd8 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:16.055016 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89e6c1-3d7b-4730-aad7-9322a969c283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:16.055896 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89e6c1-3d7b-4730-aad7-9322a969c283 None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:16.056182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89e6c1-3d7b-4730-aad7-9322a969c283 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:16.056458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89e6c1-3d7b-4730-aad7-9322a969c283 None admin] PATH_INFO: `/v3/projects/a3cabc6461bf4f29bb9c8973bdade54e/users/5f0af4c16db14e1f85719d26eb1aab68/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:16.058353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b89e6c1-3d7b-4730-aad7-9322a969c283 None admin] RBAC: Authorizing `identity:create_grant(project_id=a3cabc6461bf4f29bb9c8973bdade54e, user_id=5f0af4c16db14e1f85719d26eb1aab68, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:16.059878 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b89e6c1-3d7b-4730-aad7-9322a969c283 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:16.072699 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 610/1192] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:25:15 2022] PUT /identity/v3/projects/a3cabc6461bf4f29bb9c8973bdade54e/users/5f0af4c16db14e1f85719d26eb1aab68/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:25:18.126138 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53672319-8d0d-44a1-838d-8ed4664a547a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:18.126573 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53672319-8d0d-44a1-838d-8ed4664a547a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:18.126735 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53672319-8d0d-44a1-838d-8ed4664a547a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:18.128115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 583/1193] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:25:18.133998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c9374-f9d9-4d06-b279-d4e6ace86fb0 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:18.134409 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c9374-f9d9-4d06-b279-d4e6ace86fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:18.134564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c9374-f9d9-4d06-b279-d4e6ace86fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:18.151813 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-fc8c9374-f9d9-4d06-b279-d4e6ace86fb0 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:25:18.152990 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-fc8c9374-f9d9-4d06-b279-d4e6ace86fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:25:18.169658 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:18.219850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 611/1194] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:25:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:25:18.244568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53672319-8d0d-44a1-838d-8ed4664a547a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:18.343260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53672319-8d0d-44a1-838d-8ed4664a547a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:18.376764 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998e5e5-c8f4-49c8-8658-c274dd7017e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:18.377484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998e5e5-c8f4-49c8-8658-c274dd7017e0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:18.377767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998e5e5-c8f4-49c8-8658-c274dd7017e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:18.378046 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998e5e5-c8f4-49c8-8658-c274dd7017e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:18.388383 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998e5e5-c8f4-49c8-8658-c274dd7017e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:25:18.396225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-2998e5e5-c8f4-49c8-8658-c274dd7017e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:25:18.396225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:25:18.396225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:25:18.396225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:25:18.396225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:25:18.396225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:25:18.396225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:25:18.396225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:25:18.396225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:25:18.396225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:25:18.396225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:25:18.396225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:25:18.396225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:25:18.396225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:25:18.396225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:25:18.396225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:25:18.397349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:25:18.397349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:25:18.397349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:25:18.397349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:25:18.397349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:25:18.397349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:25:18.397349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:25:18.397349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:25:18.397349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:25:18.397349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:25:18.397349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:25:18.397349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:25:18.397349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:25:18.397349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:25:18.397349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:25:18.397349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:25:18.398388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:25:18.398388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:25:18.398388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:25:18.398388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:25:18.398388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:25:18.398388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:25:18.398388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:25:18.398388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:25:18.398388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:25:18.398388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:25:18.398388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:25:18.398388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:25:18.398388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 584/1195] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:25:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:25:18.403781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c9374-f9d9-4d06-b279-d4e6ace86fb0 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:18.497525 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c9374-f9d9-4d06-b279-d4e6ace86fb0 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:18.530657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d91c7-12d3-42db-a9db-8716e7eef2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:18.531338 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d91c7-12d3-42db-a9db-8716e7eef2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:18.531625 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d91c7-12d3-42db-a9db-8716e7eef2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:18.531902 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d91c7-12d3-42db-a9db-8716e7eef2c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:18.532634 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d91c7-12d3-42db-a9db-8716e7eef2c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:25:18.532941 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d91c7-12d3-42db-a9db-8716e7eef2c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:18.534294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d91c7-12d3-42db-a9db-8716e7eef2c5 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:18.542325 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 612/1196] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:25:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:25:18.547906 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998e5e5-c8f4-49c8-8658-c274dd7017e0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:18.604641 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:18.645939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998e5e5-c8f4-49c8-8658-c274dd7017e0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:18.678264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99258dab-ac54-4285-8701-374733abf96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:18.679155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99258dab-ac54-4285-8701-374733abf96f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:18.679445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99258dab-ac54-4285-8701-374733abf96f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:18.679725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99258dab-ac54-4285-8701-374733abf96f None admin] PATH_INFO: `/v3/users/5f0af4c16db14e1f85719d26eb1aab68` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:18.680907 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99258dab-ac54-4285-8701-374733abf96f None admin] RBAC: Authorizing `identity:get_user(user_id=5f0af4c16db14e1f85719d26eb1aab68)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:18.682332 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99258dab-ac54-4285-8701-374733abf96f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:18.683985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 585/1197] 10.222.0.43 () {56 vars in 1177 bytes} [Thu Jun 23 22:25:18 2022] GET /identity/v3/users/5f0af4c16db14e1f85719d26eb1aab68 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:25:18.689581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d91c7-12d3-42db-a9db-8716e7eef2c5 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:18.706275 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:18.789764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d91c7-12d3-42db-a9db-8716e7eef2c5 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:18.822639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9488612-e7d1-4be5-b2de-95317f46077f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:18.823306 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9488612-e7d1-4be5-b2de-95317f46077f None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:18.823597 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9488612-e7d1-4be5-b2de-95317f46077f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:18.823879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9488612-e7d1-4be5-b2de-95317f46077f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:18.890138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9488612-e7d1-4be5-b2de-95317f46077f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:18.891607 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9488612-e7d1-4be5-b2de-95317f46077f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:18.906319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:18.986421 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 613/1198] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:25:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:25:18.992770 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99258dab-ac54-4285-8701-374733abf96f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:19.097052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99258dab-ac54-4285-8701-374733abf96f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:19.121263 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:19.132048 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fbfad-9831-44df-9bcd-2086baeb4297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:19.133034 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fbfad-9831-44df-9bcd-2086baeb4297 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:19.133320 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fbfad-9831-44df-9bcd-2086baeb4297 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:19.133599 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fbfad-9831-44df-9bcd-2086baeb4297 None admin] PATH_INFO: `/v3/projects/a3cabc6461bf4f29bb9c8973bdade54e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:19.134800 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3fbfad-9831-44df-9bcd-2086baeb4297 None admin] RBAC: Authorizing `identity:get_project(project_id=a3cabc6461bf4f29bb9c8973bdade54e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:19.136604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3fbfad-9831-44df-9bcd-2086baeb4297 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:19.138590 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 586/1199] 10.222.0.43 () {56 vars in 1186 bytes} [Thu Jun 23 22:25:18 2022] GET /identity/v3/projects/a3cabc6461bf4f29bb9c8973bdade54e => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:25:19.144642 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9488612-e7d1-4be5-b2de-95317f46077f None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:25:19.240870 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9488612-e7d1-4be5-b2de-95317f46077f None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:25:19.254083 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:19.273895 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59547c-2392-4326-99af-2daf244a56ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:25:19.274799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59547c-2392-4326-99af-2daf244a56ed None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:25:19.275113 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59547c-2392-4326-99af-2daf244a56ed None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:25:19.275393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59547c-2392-4326-99af-2daf244a56ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:25:19.276213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59547c-2392-4326-99af-2daf244a56ed None admin] RBAC: Adding query filter params (role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.project.id=a3cabc6461bf4f29bb9c8973bdade54e, user.id=5f0af4c16db14e1f85719d26eb1aab68) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:25:19.276527 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59547c-2392-4326-99af-2daf244a56ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:25:19.277947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59547c-2392-4326-99af-2daf244a56ed None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:25:19.283572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 614/1200] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:25:19 2022] GET /identity/v3/role_assignments?user.id=5f0af4c16db14e1f85719d26eb1aab68&scope.project.id=a3cabc6461bf4f29bb9c8973bdade54e&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:25:29.126282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:25:29.281005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=11 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:14.633904 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a6ac7-af82-4b4c-87f3-309ac944bba7 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:14.634503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a6ac7-af82-4b4c-87f3-309ac944bba7 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:14.634503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a6ac7-af82-4b4c-87f3-309ac944bba7 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:14.636197 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 587/1201] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:14.641640 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187174ef-9ba9-438e-80b2-fc2f1556d428 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:14.642067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187174ef-9ba9-438e-80b2-fc2f1556d428 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:14.642201 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187174ef-9ba9-438e-80b2-fc2f1556d428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:14.663341 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-187174ef-9ba9-438e-80b2-fc2f1556d428 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:14.665699 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-187174ef-9ba9-438e-80b2-fc2f1556d428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:14.767243 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 615/1202] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:26:14.799035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a6ac7-af82-4b4c-87f3-309ac944bba7 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:14.951804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a6ac7-af82-4b4c-87f3-309ac944bba7 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:14.996774 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1b07e-d1e3-4174-9f8d-a47f1594f5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:14.997587 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1b07e-d1e3-4174-9f8d-a47f1594f5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:14.997932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1b07e-d1e3-4174-9f8d-a47f1594f5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:14.998223 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1b07e-d1e3-4174-9f8d-a47f1594f5a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:15.010475 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1b07e-d1e3-4174-9f8d-a47f1594f5a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:15.012015 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1b07e-d1e3-4174-9f8d-a47f1594f5a4 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:15.034285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-f8e1b07e-d1e3-4174-9f8d-a47f1594f5a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:15.034285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:15.034285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:15.034285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:15.034285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:15.034285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:15.034285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:15.034285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:15.034285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:15.034285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:15.034285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:15.034285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:15.034285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:26:15.034285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:26:15.034285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:26:15.036167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:26:15.036167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:15.036167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:15.036167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:15.036167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:15.036167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:15.036167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:15.036167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:15.036167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:15.036167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:15.036167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:15.036167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:15.036167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:15.036167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:15.036167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:15.036167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:15.036167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:15.037151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:26:15.037151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:26:15.037151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:15.037151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:26:15.037151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 588/1203] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:26:15.043928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187174ef-9ba9-438e-80b2-fc2f1556d428 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:15.174182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187174ef-9ba9-438e-80b2-fc2f1556d428 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:15.213638 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:15.221643 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b5d08-5e9a-461e-8394-044aa8206cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:15.222340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b5d08-5e9a-461e-8394-044aa8206cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:15.222729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b5d08-5e9a-461e-8394-044aa8206cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:15.223118 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b5d08-5e9a-461e-8394-044aa8206cd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:15.224439 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b5d08-5e9a-461e-8394-044aa8206cd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:15.224735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b5d08-5e9a-461e-8394-044aa8206cd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:15.226119 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b5d08-5e9a-461e-8394-044aa8206cd4 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:15.240499 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 616/1204] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:15.246407 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1b07e-d1e3-4174-9f8d-a47f1594f5a4 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:15.312426 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:15.389035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1b07e-d1e3-4174-9f8d-a47f1594f5a4 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:15.457145 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f331768-f853-4cbc-90b0-fc48fa427f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:15.458325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f331768-f853-4cbc-90b0-fc48fa427f05 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:15.458698 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f331768-f853-4cbc-90b0-fc48fa427f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:15.459038 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f331768-f853-4cbc-90b0-fc48fa427f05 None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:15.460108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f331768-f853-4cbc-90b0-fc48fa427f05 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:15.461927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f331768-f853-4cbc-90b0-fc48fa427f05 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:15.493583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 589/1205] 10.222.0.43 () {60 vars in 1132 bytes} [Thu Jun 23 22:26:15 2022] POST /identity/v3/users => generated 251 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:26:17.772295 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36562709-38c2-4bab-b168-9242f5438d4f None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:17.772857 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36562709-38c2-4bab-b168-9242f5438d4f None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:17.772857 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36562709-38c2-4bab-b168-9242f5438d4f None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:17.774430 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 617/1206] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:26:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:17.781669 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5f226-01e2-4a4c-9061-59986c5d0dd4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:17.782097 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5f226-01e2-4a4c-9061-59986c5d0dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:17.782283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5f226-01e2-4a4c-9061-59986c5d0dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:17.800532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-edc5f226-01e2-4a4c-9061-59986c5d0dd4 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:17.801999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-edc5f226-01e2-4a4c-9061-59986c5d0dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:17.887031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 590/1207] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:26:17.912894 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36562709-38c2-4bab-b168-9242f5438d4f None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:18.028918 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36562709-38c2-4bab-b168-9242f5438d4f None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:18.071745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891da521-b518-415d-9f5c-d98c54bf8d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:18.072582 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891da521-b518-415d-9f5c-d98c54bf8d7c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:18.072833 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891da521-b518-415d-9f5c-d98c54bf8d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:18.073168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891da521-b518-415d-9f5c-d98c54bf8d7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:18.086642 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891da521-b518-415d-9f5c-d98c54bf8d7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:18.095294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-891da521-b518-415d-9f5c-d98c54bf8d7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:26:18.095294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:18.095294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:18.095294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:18.095294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:18.095294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:18.095294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:18.095294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:18.095294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:18.095294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:18.095294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:18.095294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:18.095294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:18.095294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:18.095294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:18.095294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:18.096469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:18.096469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:18.096469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:18.096469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:18.096469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:18.096469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:18.096469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:18.096469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:18.096469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:18.096469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:18.096469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:18.096469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:18.096469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:18.096469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:18.096469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:18.096469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:18.097677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:18.097677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:18.097677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:18.097677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:18.097677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:18.097677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:18.097677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:18.097677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:18.097677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:18.097677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:18.097677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:26:18.097677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:18.097677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 618/1208] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:26:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:26:18.103304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5f226-01e2-4a4c-9061-59986c5d0dd4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:18.149952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:18.221239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5f226-01e2-4a4c-9061-59986c5d0dd4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:18.265942 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fcbb4f-602c-4eef-816d-1d38eccf4fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:18.266642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcbb4f-602c-4eef-816d-1d38eccf4fbe None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:18.266952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcbb4f-602c-4eef-816d-1d38eccf4fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:18.267255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcbb4f-602c-4eef-816d-1d38eccf4fbe None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:18.268008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fcbb4f-602c-4eef-816d-1d38eccf4fbe None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:18.268322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fcbb4f-602c-4eef-816d-1d38eccf4fbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:18.271767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fcbb4f-602c-4eef-816d-1d38eccf4fbe None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:18.287428 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 591/1209] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:26:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:18.288512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891da521-b518-415d-9f5c-d98c54bf8d7c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:18.344894 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:18.407646 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891da521-b518-415d-9f5c-d98c54bf8d7c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:18.449836 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fabd78-cb40-47ae-8ad9-a2e3d4f71468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:18.450657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fabd78-cb40-47ae-8ad9-a2e3d4f71468 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:18.450967 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fabd78-cb40-47ae-8ad9-a2e3d4f71468 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:18.451252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fabd78-cb40-47ae-8ad9-a2e3d4f71468 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:18.462849 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fabd78-cb40-47ae-8ad9-a2e3d4f71468 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:18.464333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fabd78-cb40-47ae-8ad9-a2e3d4f71468 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:18.482429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-73fabd78-cb40-47ae-8ad9-a2e3d4f71468 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:18.482429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:18.482429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:18.482429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:18.482429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:18.482429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:18.482429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:18.482429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:18.482429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:18.482429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:18.482429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:18.482429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:18.482429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:26:18.482429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:26:18.482429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:26:18.483626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:26:18.483626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:18.483626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:18.483626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:18.483626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:18.483626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:18.483626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:18.483626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:18.483626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:18.483626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:18.483626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:18.483626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:18.483626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:18.483626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:18.483626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:18.483626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:18.483626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:18.488177 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:26:18.488177 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:26:18.488177 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:18.488177 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:18.488177 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 619/1210] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:26:18.489774 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fcbb4f-602c-4eef-816d-1d38eccf4fbe None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:18.597982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:18.601979 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fcbb4f-602c-4eef-816d-1d38eccf4fbe None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:18.643117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb1e80-3183-4d2f-b95a-691e8982944c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:18.644335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb1e80-3183-4d2f-b95a-691e8982944c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:18.644335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb1e80-3183-4d2f-b95a-691e8982944c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:18.644500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb1e80-3183-4d2f-b95a-691e8982944c None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:18.645791 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb1e80-3183-4d2f-b95a-691e8982944c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:18.646267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb1e80-3183-4d2f-b95a-691e8982944c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:18.647831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedb1e80-3183-4d2f-b95a-691e8982944c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:18.661073 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 592/1211] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:18.667022 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fabd78-cb40-47ae-8ad9-a2e3d4f71468 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:18.786856 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fabd78-cb40-47ae-8ad9-a2e3d4f71468 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:18.788600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:18.835591 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da0f72-3710-4c4b-a758-bfb5fe59b726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:18.836673 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da0f72-3710-4c4b-a758-bfb5fe59b726 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:18.837018 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da0f72-3710-4c4b-a758-bfb5fe59b726 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:18.837499 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da0f72-3710-4c4b-a758-bfb5fe59b726 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:18.850516 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da0f72-3710-4c4b-a758-bfb5fe59b726 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:18.850516 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da0f72-3710-4c4b-a758-bfb5fe59b726 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:18.871947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-c6da0f72-3710-4c4b-a758-bfb5fe59b726 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:26:18.871947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:18.871947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:18.871947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:18.871947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:18.871947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:18.871947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:18.871947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:18.871947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:18.871947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:18.871947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:18.871947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:18.871947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:26:18.871947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:26:18.871947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:26:18.871947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:26:18.874578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:18.874578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:18.874578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:26:18.874578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:26:18.874578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:26:18.874578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:26:18.874578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:18.874578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:18.874578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:18.874578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:18.874578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:18.874578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:18.874578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:18.874578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:18.874578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:18.874578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:18.874578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:18.874578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:18.876099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:18.876099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:18.876099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:26:18.876099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:26:18.876099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:26:18.876099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:26:18.876099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:26:18.876099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:26:18.876099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:26:18.876099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:18.876099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 620/1212] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:18.881439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb1e80-3183-4d2f-b95a-691e8982944c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:18.996539 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb1e80-3183-4d2f-b95a-691e8982944c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:19.039015 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822f664-f2b7-4894-8838-d5a0b957d3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:19.039806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822f664-f2b7-4894-8838-d5a0b957d3df None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:19.040215 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822f664-f2b7-4894-8838-d5a0b957d3df None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:19.040528 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822f664-f2b7-4894-8838-d5a0b957d3df None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:19.041610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822f664-f2b7-4894-8838-d5a0b957d3df None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:19.041851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822f664-f2b7-4894-8838-d5a0b957d3df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:19.043603 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822f664-f2b7-4894-8838-d5a0b957d3df None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:19.064273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 593/1213] 10.222.0.43 () {56 vars in 1143 bytes} [Thu Jun 23 22:26:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:19.071364 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da0f72-3710-4c4b-a758-bfb5fe59b726 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:19.200021 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da0f72-3710-4c4b-a758-bfb5fe59b726 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:19.243801 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054aa7c2-b5e5-4433-98d2-4d5b062c459b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:19.244920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054aa7c2-b5e5-4433-98d2-4d5b062c459b None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:19.245273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054aa7c2-b5e5-4433-98d2-4d5b062c459b None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:19.245588 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054aa7c2-b5e5-4433-98d2-4d5b062c459b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:19.282154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:19.331179 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054aa7c2-b5e5-4433-98d2-4d5b062c459b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:19.333488 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054aa7c2-b5e5-4433-98d2-4d5b062c459b None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:19.460809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 621/1214] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 22:26:19.468024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822f664-f2b7-4894-8838-d5a0b957d3df None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:19.497583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:19.586897 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822f664-f2b7-4894-8838-d5a0b957d3df None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:19.629045 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b3de5-7374-45a9-b31d-af938fee177e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:19.629970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b3de5-7374-45a9-b31d-af938fee177e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:19.630271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b3de5-7374-45a9-b31d-af938fee177e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:19.630680 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b3de5-7374-45a9-b31d-af938fee177e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:19.642038 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b3de5-7374-45a9-b31d-af938fee177e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:19.643600 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b3de5-7374-45a9-b31d-af938fee177e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:19.663725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-0d6b3de5-7374-45a9-b31d-af938fee177e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:19.663725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:19.663725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:19.663725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:19.663725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:19.663725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:19.663725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:19.663725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:19.663725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:19.663725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:19.663725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:19.663725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:19.663725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:26:19.663725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:26:19.663725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:26:19.664910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:26:19.664910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:19.664910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:19.664910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:19.664910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:19.664910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:19.664910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:19.664910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:19.664910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:19.664910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:19.664910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:19.664910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:19.664910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:19.664910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:19.664910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:19.664910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:19.664910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:19.665837 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:26:19.665837 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:26:19.665837 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:19.665837 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:26:19.665837 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 594/1215] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:19.671539 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054aa7c2-b5e5-4433-98d2-4d5b062c459b None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:19.752168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:19.797054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054aa7c2-b5e5-4433-98d2-4d5b062c459b None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:19.840662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e195ea7-e370-4e8c-8060-e95c8c9bb91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:19.841383 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e195ea7-e370-4e8c-8060-e95c8c9bb91f None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:19.841731 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e195ea7-e370-4e8c-8060-e95c8c9bb91f None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:19.842055 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e195ea7-e370-4e8c-8060-e95c8c9bb91f None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:19.843063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e195ea7-e370-4e8c-8060-e95c8c9bb91f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:19.843473 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e195ea7-e370-4e8c-8060-e95c8c9bb91f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:19.845312 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e195ea7-e370-4e8c-8060-e95c8c9bb91f None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:19.867552 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 622/1216] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:19.868804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b3de5-7374-45a9-b31d-af938fee177e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:19.953091 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:19.993486 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b3de5-7374-45a9-b31d-af938fee177e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:20.034259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0aadfb-4691-434d-b1aa-9676c77d4acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:20.035020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0aadfb-4691-434d-b1aa-9676c77d4acf None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:20.035306 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0aadfb-4691-434d-b1aa-9676c77d4acf None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:20.035680 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0aadfb-4691-434d-b1aa-9676c77d4acf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:20.043394 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0aadfb-4691-434d-b1aa-9676c77d4acf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:20.044908 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0aadfb-4691-434d-b1aa-9676c77d4acf None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:20.060745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-bc0aadfb-4691-434d-b1aa-9676c77d4acf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:26:20.060745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:20.060745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:20.060745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:20.060745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:20.060745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:20.060745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:20.060745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:20.060745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:20.060745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:20.060745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:20.060745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:20.060745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:26:20.060745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:26:20.060745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:26:20.061866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:26:20.061866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:20.061866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:20.061866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:20.061866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:20.061866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:20.061866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:20.061866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:20.061866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:20.061866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:20.061866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:20.061866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:20.061866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:20.061866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:20.061866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:20.061866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:20.061866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:20.062795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:26:20.062795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:26:20.062795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:26:20.062795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:26:20.062795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:26:20.062795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:26:20.062795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:26:20.062795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:26:20.062795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 595/1217] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:26:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:20.069242 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e195ea7-e370-4e8c-8060-e95c8c9bb91f None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:20.198680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e195ea7-e370-4e8c-8060-e95c8c9bb91f None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:20.224037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:20.240555 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bbd8b-ab77-4207-80d9-54c213c3ddec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:20.241405 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bbd8b-ab77-4207-80d9-54c213c3ddec None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:20.241732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bbd8b-ab77-4207-80d9-54c213c3ddec None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:20.242033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bbd8b-ab77-4207-80d9-54c213c3ddec None admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:20.242823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bbd8b-ab77-4207-80d9-54c213c3ddec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:20.243209 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bbd8b-ab77-4207-80d9-54c213c3ddec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:20.244666 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bbd8b-ab77-4207-80d9-54c213c3ddec None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:20.259473 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 623/1218] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:26:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:20.265727 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0aadfb-4691-434d-b1aa-9676c77d4acf None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:20.382488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0aadfb-4691-434d-b1aa-9676c77d4acf None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:20.421039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:20.430837 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da75753-17b0-436e-abe1-32d73be8053b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:20.431584 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da75753-17b0-436e-abe1-32d73be8053b None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:20.432745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da75753-17b0-436e-abe1-32d73be8053b None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:20.432745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da75753-17b0-436e-abe1-32d73be8053b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:20.433023 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da75753-17b0-436e-abe1-32d73be8053b None admin] RBAC: Adding query filter params (role.id=2a71a120d6e34544aa5a2031ddf65ec8, scope.project.id=56da0975019c44409b53be473131143a, user.id=101864afd59543e699189917391e6ab1) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:20.433314 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da75753-17b0-436e-abe1-32d73be8053b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:20.435140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da75753-17b0-436e-abe1-32d73be8053b None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:20.441210 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 596/1219] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:20 2022] GET /identity/v3/role_assignments?user.id=101864afd59543e699189917391e6ab1&scope.project.id=56da0975019c44409b53be473131143a&role.id=2a71a120d6e34544aa5a2031ddf65ec8 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:22.526480 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a23a1-0b93-4873-a4fe-003946718f2d None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:22.527039 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a23a1-0b93-4873-a4fe-003946718f2d None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:22.527158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a23a1-0b93-4873-a4fe-003946718f2d None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:22.528492 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 624/1220] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:22.533828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78329e7-b328-4199-9ace-e6df9433b0f6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:22.534230 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78329e7-b328-4199-9ace-e6df9433b0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:22.534385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78329e7-b328-4199-9ace-e6df9433b0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:22.551208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a78329e7-b328-4199-9ace-e6df9433b0f6 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:22.553095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a78329e7-b328-4199-9ace-e6df9433b0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:22.626383 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 597/1221] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:26:22.651214 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a23a1-0b93-4873-a4fe-003946718f2d None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:22.759919 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a23a1-0b93-4873-a4fe-003946718f2d None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:22.800590 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958717d-bfef-47df-b5ed-a279a77900ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:22.801355 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958717d-bfef-47df-b5ed-a279a77900ce None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:22.801636 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958717d-bfef-47df-b5ed-a279a77900ce None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:22.801915 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958717d-bfef-47df-b5ed-a279a77900ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:22.812298 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958717d-bfef-47df-b5ed-a279a77900ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:22.820815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-7958717d-bfef-47df-b5ed-a279a77900ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:26:22.820815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:22.820815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:22.820815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:22.820815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:22.820815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:22.820815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:22.820815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:22.820815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:22.820815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:22.820815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:22.820815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:22.820815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:22.820815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:22.820815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:22.820815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:22.821815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:22.821815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:22.821815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:22.821815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:22.821815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:22.821815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:22.821815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:22.821815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:22.821815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:22.821815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:22.821815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:22.821815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:22.821815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:22.821815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:22.821815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:22.821815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:22.822653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:22.822653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:22.822653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:22.822653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:22.822653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:22.822653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:22.822653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:22.822653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:22.822653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:22.822653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:22.822653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:26:22.822653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:22.822653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 625/1222] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:26:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:22.828417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78329e7-b328-4199-9ace-e6df9433b0f6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:22.945035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78329e7-b328-4199-9ace-e6df9433b0f6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:22.975210 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:22.986224 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8e73e-6994-4b48-9232-7d4093d194a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:22.986904 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8e73e-6994-4b48-9232-7d4093d194a5 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:22.987214 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8e73e-6994-4b48-9232-7d4093d194a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:22.987494 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8e73e-6994-4b48-9232-7d4093d194a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:22.988237 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8e73e-6994-4b48-9232-7d4093d194a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:22.990628 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8e73e-6994-4b48-9232-7d4093d194a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:22.990628 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8e73e-6994-4b48-9232-7d4093d194a5 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:22.998421 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 598/1223] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:26:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:23.004299 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958717d-bfef-47df-b5ed-a279a77900ce None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:23.051977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:23.111467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958717d-bfef-47df-b5ed-a279a77900ce None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:23.146753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87108d02-cdaa-4541-9d51-97820bd4e7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:23.147582 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87108d02-cdaa-4541-9d51-97820bd4e7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:23.147869 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87108d02-cdaa-4541-9d51-97820bd4e7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:23.148145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87108d02-cdaa-4541-9d51-97820bd4e7d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:23.159033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87108d02-cdaa-4541-9d51-97820bd4e7d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:23.161042 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87108d02-cdaa-4541-9d51-97820bd4e7d0 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:23.178983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-87108d02-cdaa-4541-9d51-97820bd4e7d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:23.178983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:23.178983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:23.178983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:23.178983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:23.178983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:23.178983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:23.178983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:23.178983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:23.178983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:23.178983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:23.178983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:23.178983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:26:23.178983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:26:23.178983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:26:23.180100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:26:23.180100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:23.180100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:23.180100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:23.180100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:23.180100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:23.180100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:23.180100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:23.180100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:23.180100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:23.180100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:23.180100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:23.180100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:23.180100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:23.180100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:23.180100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:23.180100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:23.181386 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:26:23.181386 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:26:23.181386 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:23.181386 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:23.181386 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 626/1224] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:23.186009 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8e73e-6994-4b48-9232-7d4093d194a5 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:23.299004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8e73e-6994-4b48-9232-7d4093d194a5 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:23.335044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f0b19-7136-48ac-a233-cea46327c534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:23.335644 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f0b19-7136-48ac-a233-cea46327c534 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:23.335925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f0b19-7136-48ac-a233-cea46327c534 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:23.336203 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f0b19-7136-48ac-a233-cea46327c534 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:23.336917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f0b19-7136-48ac-a233-cea46327c534 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:23.337224 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f0b19-7136-48ac-a233-cea46327c534 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:23.338594 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f0b19-7136-48ac-a233-cea46327c534 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:23.350684 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 599/1225] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:23.356883 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87108d02-cdaa-4541-9d51-97820bd4e7d0 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:23.462601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:23.469919 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87108d02-cdaa-4541-9d51-97820bd4e7d0 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:23.506374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeed7c9-7865-4fb9-9710-ac9690e89c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:23.507375 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeed7c9-7865-4fb9-9710-ac9690e89c12 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:23.507662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeed7c9-7865-4fb9-9710-ac9690e89c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:23.507949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeed7c9-7865-4fb9-9710-ac9690e89c12 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:23.516063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeed7c9-7865-4fb9-9710-ac9690e89c12 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:23.517510 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeed7c9-7865-4fb9-9710-ac9690e89c12 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:23.533749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-9eeed7c9-7865-4fb9-9710-ac9690e89c12 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:26:23.533749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:23.533749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:23.533749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:23.533749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:23.533749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:23.533749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:23.533749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:23.533749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:23.533749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:23.533749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:23.533749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:23.533749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:26:23.533749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:26:23.533749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:26:23.533749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:26:23.535534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:23.535534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:23.535534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:26:23.535534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:26:23.535534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:26:23.535534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:26:23.535534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:23.535534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:23.535534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:23.535534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:23.535534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:23.535534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:23.535534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:23.535534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:23.535534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:23.535534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:23.535534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:23.535534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:23.537367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:23.537367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:23.537367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:26:23.537367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:26:23.537367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:26:23.537367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:26:23.537367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:26:23.537367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:26:23.537367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:26:23.537367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:23.537367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 627/1226] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:26:23.541148 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f0b19-7136-48ac-a233-cea46327c534 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:23.594321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:23.655676 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f0b19-7136-48ac-a233-cea46327c534 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:23.693013 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809dfbaa-c14b-448a-9f9d-742e0d23552a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:23.693684 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dfbaa-c14b-448a-9f9d-742e0d23552a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:23.693967 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dfbaa-c14b-448a-9f9d-742e0d23552a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:23.694260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dfbaa-c14b-448a-9f9d-742e0d23552a None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:23.695125 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809dfbaa-c14b-448a-9f9d-742e0d23552a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:23.695474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809dfbaa-c14b-448a-9f9d-742e0d23552a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:23.696867 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809dfbaa-c14b-448a-9f9d-742e0d23552a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:23.715013 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 600/1227] 10.222.0.43 () {56 vars in 1143 bytes} [Thu Jun 23 22:26:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:23.720911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeed7c9-7865-4fb9-9710-ac9690e89c12 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:23.827551 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeed7c9-7865-4fb9-9710-ac9690e89c12 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:23.868308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31bb29-813d-4958-a063-89b69936c2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:23.869040 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31bb29-813d-4958-a063-89b69936c2db None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:23.869348 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31bb29-813d-4958-a063-89b69936c2db None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:23.869598 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31bb29-813d-4958-a063-89b69936c2db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:23.882914 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31bb29-813d-4958-a063-89b69936c2db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:23.884470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31bb29-813d-4958-a063-89b69936c2db None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:23.903850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-8e31bb29-813d-4958-a063-89b69936c2db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:23.903850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:23.903850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:23.903850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:23.903850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:23.903850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:23.903850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:23.903850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:23.903850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:23.903850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:23.903850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:23.903850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:23.903850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:26:23.903850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:26:23.903850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:26:23.905198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:26:23.905198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:23.905198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:23.905198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:23.905198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:23.905198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:23.905198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:23.905198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:23.905198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:23.905198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:23.905198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:23.905198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:23.905198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:23.905198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:23.905198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:23.905198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:23.905198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:23.906392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:26:23.906392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:26:23.906392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:23.906392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:23.906392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 628/1228] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:23.911614 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809dfbaa-c14b-448a-9f9d-742e0d23552a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:24.023799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809dfbaa-c14b-448a-9f9d-742e0d23552a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:24.026425 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:24.064699 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e3b16-9ea5-4a87-8d68-18a7a67e0c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:24.065744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e3b16-9ea5-4a87-8d68-18a7a67e0c24 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:24.066006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e3b16-9ea5-4a87-8d68-18a7a67e0c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:24.066372 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e3b16-9ea5-4a87-8d68-18a7a67e0c24 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:24.067293 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e3b16-9ea5-4a87-8d68-18a7a67e0c24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:24.067607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e3b16-9ea5-4a87-8d68-18a7a67e0c24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:24.068972 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e3b16-9ea5-4a87-8d68-18a7a67e0c24 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:24.080246 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 601/1229] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:24.086465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31bb29-813d-4958-a063-89b69936c2db None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:24.096860 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:24.202777 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31bb29-813d-4958-a063-89b69936c2db None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:24.239964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f24002-23cb-4401-9367-a99ef2f73ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:24.240801 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f24002-23cb-4401-9367-a99ef2f73ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:24.241090 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f24002-23cb-4401-9367-a99ef2f73ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:24.241371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f24002-23cb-4401-9367-a99ef2f73ba4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:24.248147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f24002-23cb-4401-9367-a99ef2f73ba4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:24.249627 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f24002-23cb-4401-9367-a99ef2f73ba4 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:24.264128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-b5f24002-23cb-4401-9367-a99ef2f73ba4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:26:24.264128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:24.264128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:24.264128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:24.264128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:24.264128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:24.264128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:24.264128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:24.264128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:24.264128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:24.264128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:24.264128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:24.264128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:26:24.264128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:26:24.264128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:26:24.265376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:26:24.265376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:24.265376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:24.265376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:24.265376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:24.265376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:24.265376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:24.265376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:24.265376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:24.265376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:24.265376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:24.265376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:24.265376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:24.265376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:24.265376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:24.265376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:24.265376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:24.266464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:26:24.266464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:26:24.266464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:26:24.266464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:26:24.266464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:26:24.266464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:26:24.266464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:26:24.266464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:24.270640 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 629/1230] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:26:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:26:24.271689 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e3b16-9ea5-4a87-8d68-18a7a67e0c24 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:24.381643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e3b16-9ea5-4a87-8d68-18a7a67e0c24 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:24.421452 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b5830-6f24-474d-99ba-4f0dc7397f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:24.422109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b5830-6f24-474d-99ba-4f0dc7397f11 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:24.422463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b5830-6f24-474d-99ba-4f0dc7397f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:24.422746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b5830-6f24-474d-99ba-4f0dc7397f11 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:24.423564 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b5830-6f24-474d-99ba-4f0dc7397f11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:24.423901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b5830-6f24-474d-99ba-4f0dc7397f11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:24.425555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b5830-6f24-474d-99ba-4f0dc7397f11 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:24.438596 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 602/1231] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:26:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:24.445167 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f24002-23cb-4401-9367-a99ef2f73ba4 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:24.517166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:24.559860 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f24002-23cb-4401-9367-a99ef2f73ba4 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:24.597546 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783a2ea-e8fd-47f7-96df-4f09a437e793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:24.599114 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783a2ea-e8fd-47f7-96df-4f09a437e793 None admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:24.599453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783a2ea-e8fd-47f7-96df-4f09a437e793 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:24.599799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783a2ea-e8fd-47f7-96df-4f09a437e793 None admin] PATH_INFO: `/v3/projects/56da0975019c44409b53be473131143a/users/101864afd59543e699189917391e6ab1/roles/2a71a120d6e34544aa5a2031ddf65ec8` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:24.615714 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783a2ea-e8fd-47f7-96df-4f09a437e793 None admin] RBAC: Authorizing `identity:create_grant(project_id=56da0975019c44409b53be473131143a, user_id=101864afd59543e699189917391e6ab1, role_id=2a71a120d6e34544aa5a2031ddf65ec8)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:24.617300 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783a2ea-e8fd-47f7-96df-4f09a437e793 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:24.629526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 630/1232] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:26:24 2022] PUT /identity/v3/projects/56da0975019c44409b53be473131143a/users/101864afd59543e699189917391e6ab1/roles/2a71a120d6e34544aa5a2031ddf65ec8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:26:26.828527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e5b30-37d7-46f6-a8b3-0d958a7b99fc None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:26.829090 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e5b30-37d7-46f6-a8b3-0d958a7b99fc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:26.829170 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e5b30-37d7-46f6-a8b3-0d958a7b99fc None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:26.830749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 603/1233] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:26:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:26.836938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0679ee-8a98-4a3f-8549-8f64a70ba7d0 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:26.837393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0679ee-8a98-4a3f-8549-8f64a70ba7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:26.837525 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0679ee-8a98-4a3f-8549-8f64a70ba7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:26.854960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-6e0679ee-8a98-4a3f-8549-8f64a70ba7d0 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:26.856167 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-6e0679ee-8a98-4a3f-8549-8f64a70ba7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:26.938536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 631/1234] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:26:26.966580 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e5b30-37d7-46f6-a8b3-0d958a7b99fc None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:26.997400 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:27.085708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e5b30-37d7-46f6-a8b3-0d958a7b99fc None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:27.122534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab313f4b-c242-454c-8b54-92f00152a028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:27.122534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab313f4b-c242-454c-8b54-92f00152a028 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:27.122534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab313f4b-c242-454c-8b54-92f00152a028 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:27.123084 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab313f4b-c242-454c-8b54-92f00152a028 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:27.135479 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab313f4b-c242-454c-8b54-92f00152a028 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:27.143161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-ab313f4b-c242-454c-8b54-92f00152a028 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:26:27.143161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:27.143161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:27.143161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:27.143161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:27.143161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:27.143161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:27.143161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:27.143161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:27.143161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:27.143161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:27.143161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:27.143161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:27.143161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:27.143161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:27.143161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:27.144264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:27.144264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:27.144264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:27.144264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:27.144264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:27.144264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:27.144264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:27.144264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:27.144264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:27.144264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:27.144264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:27.144264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:27.144264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:27.144264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:27.144264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:27.144264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:27.146323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:27.146323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:27.146323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:27.146323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:27.146323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:27.146323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:27.146323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:27.146323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:27.146323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:27.146323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:27.146323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 22:26:27.146323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:26:27.146323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 604/1235] 10.222.0.43 () {56 vars in 1102 bytes} [Thu Jun 23 22:26:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:27.153383 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0679ee-8a98-4a3f-8549-8f64a70ba7d0 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:27.203919 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:27.271140 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0679ee-8a98-4a3f-8549-8f64a70ba7d0 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:27.310501 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e89838-1e13-44fd-8a06-54b2927f3ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:27.311250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e89838-1e13-44fd-8a06-54b2927f3ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:27.311608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e89838-1e13-44fd-8a06-54b2927f3ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:27.311944 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e89838-1e13-44fd-8a06-54b2927f3ce7 None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:27.312845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e89838-1e13-44fd-8a06-54b2927f3ce7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:27.313168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e89838-1e13-44fd-8a06-54b2927f3ce7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:27.314672 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e89838-1e13-44fd-8a06-54b2927f3ce7 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:27.324717 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 632/1236] 10.222.0.43 () {56 vars in 1103 bytes} [Thu Jun 23 22:26:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:27.331347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab313f4b-c242-454c-8b54-92f00152a028 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:27.424323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:27.441356 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab313f4b-c242-454c-8b54-92f00152a028 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:27.479803 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8cb1d-c3ae-4929-9c2a-e6064d146877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:27.480640 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8cb1d-c3ae-4929-9c2a-e6064d146877 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:27.480853 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8cb1d-c3ae-4929-9c2a-e6064d146877 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:27.481141 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8cb1d-c3ae-4929-9c2a-e6064d146877 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:27.492433 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8cb1d-c3ae-4929-9c2a-e6064d146877 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:27.493886 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8cb1d-c3ae-4929-9c2a-e6064d146877 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:27.513116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d6e8cb1d-c3ae-4929-9c2a-e6064d146877 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:27.513116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:27.513116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:27.513116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:27.513116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:27.513116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:27.513116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:27.513116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:27.513116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:27.513116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:27.513116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:27.513116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:27.513116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:26:27.513116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:26:27.513116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:26:27.514289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:26:27.514289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:27.514289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:27.514289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:27.514289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:27.514289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:27.514289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:27.514289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:27.514289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:27.514289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:27.514289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:27.514289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:27.514289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:27.514289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:27.514289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:27.514289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:27.514289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:27.515527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:26:27.515527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:26:27.515527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:27.515527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:26:27.519360 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 605/1237] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:27.520971 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e89838-1e13-44fd-8a06-54b2927f3ce7 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:27.631212 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e89838-1e13-44fd-8a06-54b2927f3ce7 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:27.641858 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:27.673342 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2010fc0-e1ff-4d3c-a501-9ba72924b635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:27.674146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2010fc0-e1ff-4d3c-a501-9ba72924b635 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:27.679698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2010fc0-e1ff-4d3c-a501-9ba72924b635 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:27.679698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2010fc0-e1ff-4d3c-a501-9ba72924b635 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:27.679698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2010fc0-e1ff-4d3c-a501-9ba72924b635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:27.679698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2010fc0-e1ff-4d3c-a501-9ba72924b635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:27.679698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2010fc0-e1ff-4d3c-a501-9ba72924b635 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:27.694398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 633/1238] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:27.700390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8cb1d-c3ae-4929-9c2a-e6064d146877 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:27.820251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8cb1d-c3ae-4929-9c2a-e6064d146877 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:27.864017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aae973-9778-4cce-867e-cdb448312e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:27.865062 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aae973-9778-4cce-867e-cdb448312e7a None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:27.865324 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aae973-9778-4cce-867e-cdb448312e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:27.865656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aae973-9778-4cce-867e-cdb448312e7a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:27.869626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:27.876414 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aae973-9778-4cce-867e-cdb448312e7a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:27.878619 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aae973-9778-4cce-867e-cdb448312e7a None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:27.898371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-57aae973-9778-4cce-867e-cdb448312e7a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:26:27.898371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:27.898371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:27.898371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:27.898371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:27.898371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:27.898371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:27.898371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:27.898371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:27.898371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:27.898371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:27.898371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:27.898371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:26:27.898371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:26:27.898371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:26:27.898371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:26:27.901188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:27.901188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:27.901188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:26:27.901188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:26:27.901188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:26:27.901188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:26:27.901188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:27.901188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:27.901188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:27.901188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:27.901188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:27.901188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:27.901188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:27.901188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:27.901188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:27.901188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:27.901188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:27.901188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:27.903387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:27.903387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:27.903387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:26:27.903387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:26:27.903387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:26:27.903387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:26:27.903387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:26:27.903387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:26:27.903387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:26:27.903387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:26:27.903387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 606/1239] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:27.908059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2010fc0-e1ff-4d3c-a501-9ba72924b635 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:28.019033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2010fc0-e1ff-4d3c-a501-9ba72924b635 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:28.060367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30a564-7022-407f-ba3c-9a88eafced66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:28.061139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30a564-7022-407f-ba3c-9a88eafced66 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:28.061474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30a564-7022-407f-ba3c-9a88eafced66 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:28.061805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30a564-7022-407f-ba3c-9a88eafced66 None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:28.062745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30a564-7022-407f-ba3c-9a88eafced66 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:28.063127 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30a564-7022-407f-ba3c-9a88eafced66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:28.064638 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30a564-7022-407f-ba3c-9a88eafced66 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:28.084570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 634/1240] 10.222.0.43 () {56 vars in 1143 bytes} [Thu Jun 23 22:26:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:28.090681 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aae973-9778-4cce-867e-cdb448312e7a None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:28.208620 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aae973-9778-4cce-867e-cdb448312e7a None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:28.245065 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d6c83-2369-4565-81cd-2a924d4c6d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:28.245749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d6c83-2369-4565-81cd-2a924d4c6d4c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:28.246039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d6c83-2369-4565-81cd-2a924d4c6d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:28.246322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d6c83-2369-4565-81cd-2a924d4c6d4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:28.317166 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:28.324379 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d6c83-2369-4565-81cd-2a924d4c6d4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:28.325935 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1d6c83-2369-4565-81cd-2a924d4c6d4c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:28.439977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 607/1241] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 22:26:28.446426 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30a564-7022-407f-ba3c-9a88eafced66 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:28.476548 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:28.554060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30a564-7022-407f-ba3c-9a88eafced66 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:28.593366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617aede-dadf-4af6-82d0-2477762f7cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:28.594107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617aede-dadf-4af6-82d0-2477762f7cdf None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:28.594382 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617aede-dadf-4af6-82d0-2477762f7cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:28.594667 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617aede-dadf-4af6-82d0-2477762f7cdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:28.605445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617aede-dadf-4af6-82d0-2477762f7cdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:28.606976 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617aede-dadf-4af6-82d0-2477762f7cdf None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:28.625823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-2617aede-dadf-4af6-82d0-2477762f7cdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:28.625823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:28.625823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:28.625823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:28.625823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:28.625823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:28.625823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:28.625823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:28.625823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:28.625823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:28.625823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:28.625823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:28.625823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:26:28.625823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:26:28.625823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:26:28.626960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:26:28.626960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:28.626960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:28.626960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:28.626960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:28.626960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:28.626960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:28.626960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:28.626960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:28.626960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:28.626960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:28.626960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:28.626960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:28.626960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:28.626960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:28.626960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:28.626960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:28.627913 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:26:28.627913 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:26:28.627913 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:28.627913 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:28.627913 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 635/1242] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:28.633629 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d6c83-2369-4565-81cd-2a924d4c6d4c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:28.735508 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:28.743686 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1d6c83-2369-4565-81cd-2a924d4c6d4c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:28.784701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08c36a-4b50-4b5d-96a8-e58766b26a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:28.785403 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08c36a-4b50-4b5d-96a8-e58766b26a87 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:28.786432 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08c36a-4b50-4b5d-96a8-e58766b26a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:28.786432 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08c36a-4b50-4b5d-96a8-e58766b26a87 None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:28.787039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab08c36a-4b50-4b5d-96a8-e58766b26a87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:28.787396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab08c36a-4b50-4b5d-96a8-e58766b26a87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:28.789215 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab08c36a-4b50-4b5d-96a8-e58766b26a87 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:28.804612 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 608/1243] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:28.810296 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617aede-dadf-4af6-82d0-2477762f7cdf None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:28.905682 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:28.931066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617aede-dadf-4af6-82d0-2477762f7cdf None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:28.967684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a9cfe-7095-4e47-988b-3a14667837df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:28.968530 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a9cfe-7095-4e47-988b-3a14667837df None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:28.968891 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a9cfe-7095-4e47-988b-3a14667837df None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:28.969196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a9cfe-7095-4e47-988b-3a14667837df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:28.976891 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a9cfe-7095-4e47-988b-3a14667837df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:28.978351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a9cfe-7095-4e47-988b-3a14667837df None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:28.993772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-d65a9cfe-7095-4e47-988b-3a14667837df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:26:28.993772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:28.993772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:28.993772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:28.993772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:28.993772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:28.993772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:28.993772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:28.993772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:28.993772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:28.993772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:28.993772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:28.993772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:26:28.993772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:26:28.993772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:26:28.994980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:26:28.994980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:28.994980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:28.994980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:28.994980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:28.994980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:28.994980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:28.994980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:28.994980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:28.994980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:28.994980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:28.994980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:28.994980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:28.994980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:28.994980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:28.994980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:28.994980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:28.995940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:26:28.995940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:26:28.995940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:26:28.995940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:26:28.995940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:26:28.995940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:26:28.995940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:26:28.995940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:28.995940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 636/1244] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:26:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:29.001386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08c36a-4b50-4b5d-96a8-e58766b26a87 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:29.113333 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08c36a-4b50-4b5d-96a8-e58766b26a87 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:29.157425 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b406b-609b-4f44-9dde-960d9e2830bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:29.158226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b406b-609b-4f44-9dde-960d9e2830bb None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:29.158509 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b406b-609b-4f44-9dde-960d9e2830bb None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:29.158849 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b406b-609b-4f44-9dde-960d9e2830bb None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:29.159696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b406b-609b-4f44-9dde-960d9e2830bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:29.160101 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b406b-609b-4f44-9dde-960d9e2830bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:29.162384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b406b-609b-4f44-9dde-960d9e2830bb None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:29.177249 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 609/1245] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:26:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:29.184256 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a9cfe-7095-4e47-988b-3a14667837df None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:29.300324 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a9cfe-7095-4e47-988b-3a14667837df None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:29.339011 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef333e1-5521-4012-8334-a642a01b06f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:29.339676 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef333e1-5521-4012-8334-a642a01b06f9 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:29.339960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef333e1-5521-4012-8334-a642a01b06f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:29.340245 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef333e1-5521-4012-8334-a642a01b06f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:29.341077 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef333e1-5521-4012-8334-a642a01b06f9 None admin] RBAC: Adding query filter params (role.id=2a71a120d6e34544aa5a2031ddf65ec8, scope.project.id=56da0975019c44409b53be473131143a, user.id=101864afd59543e699189917391e6ab1) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:29.341394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef333e1-5521-4012-8334-a642a01b06f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:29.343006 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef333e1-5521-4012-8334-a642a01b06f9 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:29.360331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 637/1246] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:29 2022] GET /identity/v3/role_assignments?user.id=101864afd59543e699189917391e6ab1&scope.project.id=56da0975019c44409b53be473131143a&role.id=2a71a120d6e34544aa5a2031ddf65ec8 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:31.503439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ae3b1-9eda-4d9f-a965-a4b6b54b1d7a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:31.504075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ae3b1-9eda-4d9f-a965-a4b6b54b1d7a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:31.504075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ae3b1-9eda-4d9f-a965-a4b6b54b1d7a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:31.505506 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 610/1247] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:31.511092 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9ab44-ed30-4efa-8919-0585b6350ab6 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:31.511519 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9ab44-ed30-4efa-8919-0585b6350ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:31.511685 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9ab44-ed30-4efa-8919-0585b6350ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:31.516899 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:31.531387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-1aa9ab44-ed30-4efa-8919-0585b6350ab6 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:31.532604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1aa9ab44-ed30-4efa-8919-0585b6350ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:31.608026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 638/1248] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:26:31.633482 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ae3b1-9eda-4d9f-a965-a4b6b54b1d7a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:31.652922 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:31.753180 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ae3b1-9eda-4d9f-a965-a4b6b54b1d7a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:31.793105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8c8cd-9ebc-43a4-9ae7-40c43fdaa826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:31.793907 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8c8cd-9ebc-43a4-9ae7-40c43fdaa826 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:31.794200 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8c8cd-9ebc-43a4-9ae7-40c43fdaa826 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:31.794488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c8c8cd-9ebc-43a4-9ae7-40c43fdaa826 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:31.806745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c8c8cd-9ebc-43a4-9ae7-40c43fdaa826 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:31.815107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-37c8c8cd-9ebc-43a4-9ae7-40c43fdaa826 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:31.815107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:31.815107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:31.815107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:31.815107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:31.815107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:31.815107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:31.815107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:31.815107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:31.815107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:31.815107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:31.815107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:31.815107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:31.815107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:31.815107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:31.815107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:31.816265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:31.816265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:31.816265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:31.816265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:31.816265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:31.816265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:31.816265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:31.816265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:31.816265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:31.816265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:31.816265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:31.816265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:31.816265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:31.816265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:31.816265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:31.816265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:31.817161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:31.817161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:31.817161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:31.817161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:31.817161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:31.817161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:31.817161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:31.817161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:31.817161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:31.817161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:31.817161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:31.817161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:26:31.817161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 611/1249] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:26:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:31.822357 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9ab44-ed30-4efa-8919-0585b6350ab6 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:31.943242 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9ab44-ed30-4efa-8919-0585b6350ab6 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:31.979946 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:31.987773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d69b8-9491-445c-921f-244c762d9d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:31.988493 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d69b8-9491-445c-921f-244c762d9d4a None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:31.988917 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d69b8-9491-445c-921f-244c762d9d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:31.989210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d69b8-9491-445c-921f-244c762d9d4a None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:31.990156 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d69b8-9491-445c-921f-244c762d9d4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:31.990482 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d69b8-9491-445c-921f-244c762d9d4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:31.991891 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d69b8-9491-445c-921f-244c762d9d4a None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:32.001133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 639/1250] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:32.007729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8c8cd-9ebc-43a4-9ae7-40c43fdaa826 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:32.099239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:32.131027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c8c8cd-9ebc-43a4-9ae7-40c43fdaa826 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:32.168823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06573dd-edf4-4983-b5d6-e463383adaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:32.169633 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06573dd-edf4-4983-b5d6-e463383adaef None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:32.169971 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06573dd-edf4-4983-b5d6-e463383adaef None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:32.170336 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06573dd-edf4-4983-b5d6-e463383adaef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:32.183184 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06573dd-edf4-4983-b5d6-e463383adaef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:32.184922 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06573dd-edf4-4983-b5d6-e463383adaef None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:32.204597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a06573dd-edf4-4983-b5d6-e463383adaef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:32.204597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:32.204597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:32.204597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:32.204597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:32.204597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:32.204597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:32.204597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:32.204597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:32.204597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:32.204597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:32.204597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:32.204597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:26:32.204597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:26:32.204597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:26:32.205780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:26:32.205780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:32.205780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:32.205780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:32.205780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:32.205780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:32.205780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:32.205780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:32.205780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:32.205780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:32.205780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:32.205780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:32.205780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:32.205780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:32.205780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:32.205780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:32.205780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:32.206723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:26:32.206723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:26:32.206723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:32.206723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:26:32.206723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 612/1251] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 22:26:32.212691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d69b8-9491-445c-921f-244c762d9d4a None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:32.331459 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d69b8-9491-445c-921f-244c762d9d4a None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:32.368467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c9d13-76e1-42fc-8e11-c521f9fe1055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:32.369061 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c9d13-76e1-42fc-8e11-c521f9fe1055 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:32.369346 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c9d13-76e1-42fc-8e11-c521f9fe1055 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:32.369626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c9d13-76e1-42fc-8e11-c521f9fe1055 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:32.370340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c9d13-76e1-42fc-8e11-c521f9fe1055 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:32.370653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c9d13-76e1-42fc-8e11-c521f9fe1055 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:32.372059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c9d13-76e1-42fc-8e11-c521f9fe1055 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:32.383591 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 640/1252] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:32.389243 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06573dd-edf4-4983-b5d6-e463383adaef None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:32.495715 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06573dd-edf4-4983-b5d6-e463383adaef None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:32.523775 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:32.532700 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f166b8-a6f4-4cdd-b891-bbf2d6fedeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:32.533600 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f166b8-a6f4-4cdd-b891-bbf2d6fedeb1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:32.533910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f166b8-a6f4-4cdd-b891-bbf2d6fedeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:32.534234 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f166b8-a6f4-4cdd-b891-bbf2d6fedeb1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:32.542572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f166b8-a6f4-4cdd-b891-bbf2d6fedeb1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:32.544235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f166b8-a6f4-4cdd-b891-bbf2d6fedeb1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:32.560117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d8f166b8-a6f4-4cdd-b891-bbf2d6fedeb1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:26:32.560117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:32.560117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:32.560117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:32.560117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:32.560117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:32.560117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:32.560117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:32.560117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:32.560117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:32.560117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:32.560117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:32.560117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:26:32.560117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:26:32.560117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:26:32.560117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:26:32.561110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:32.561110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:32.561110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:26:32.561110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:26:32.561110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:26:32.561110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:26:32.561110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:32.561110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:32.561110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:32.561110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:32.561110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:32.561110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:32.561110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:32.561110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:32.561110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:32.561110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:32.561110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:32.561110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:32.562035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:32.562035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:32.562035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:26:32.562035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:26:32.562035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:26:32.562035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:26:32.562035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:26:32.562035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:26:32.562035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:26:32.562035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:26:32.562035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 613/1253] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:32.567959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c9d13-76e1-42fc-8e11-c521f9fe1055 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:32.631293 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:32.683158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c9d13-76e1-42fc-8e11-c521f9fe1055 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:32.720644 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61465987-b525-4c83-8482-87aca211cdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:32.721464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61465987-b525-4c83-8482-87aca211cdc1 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:32.721751 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61465987-b525-4c83-8482-87aca211cdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:32.722094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61465987-b525-4c83-8482-87aca211cdc1 None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:32.723579 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61465987-b525-4c83-8482-87aca211cdc1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:32.723912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61465987-b525-4c83-8482-87aca211cdc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:32.725368 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61465987-b525-4c83-8482-87aca211cdc1 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:32.744634 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 641/1254] 10.222.0.43 () {56 vars in 1143 bytes} [Thu Jun 23 22:26:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:32.750992 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f166b8-a6f4-4cdd-b891-bbf2d6fedeb1 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:32.860741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f166b8-a6f4-4cdd-b891-bbf2d6fedeb1 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:32.901655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8434f7-8a04-4acd-a77f-a3cff3ef0371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:32.902394 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8434f7-8a04-4acd-a77f-a3cff3ef0371 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:32.902574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8434f7-8a04-4acd-a77f-a3cff3ef0371 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:32.902863 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8434f7-8a04-4acd-a77f-a3cff3ef0371 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:32.958487 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:32.972190 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8434f7-8a04-4acd-a77f-a3cff3ef0371 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:32.974484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8434f7-8a04-4acd-a77f-a3cff3ef0371 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:33.081925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 614/1255] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 22:26:33.088473 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61465987-b525-4c83-8482-87aca211cdc1 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:33.193205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61465987-b525-4c83-8482-87aca211cdc1 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:33.206482 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:33.230139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29f524-ff4c-44d6-b0c7-75d2e64afcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:33.231006 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29f524-ff4c-44d6-b0c7-75d2e64afcc2 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:33.231327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29f524-ff4c-44d6-b0c7-75d2e64afcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:33.231610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29f524-ff4c-44d6-b0c7-75d2e64afcc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:33.243212 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f29f524-ff4c-44d6-b0c7-75d2e64afcc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:33.244748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f29f524-ff4c-44d6-b0c7-75d2e64afcc2 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:33.264059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-4f29f524-ff4c-44d6-b0c7-75d2e64afcc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:33.264059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:33.264059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:33.264059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:33.264059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:33.264059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:33.264059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:33.264059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:33.264059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:33.264059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:33.264059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:33.264059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:33.264059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:26:33.264059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:26:33.264059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:26:33.265202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:26:33.265202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:33.265202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:33.265202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:33.265202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:33.265202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:33.265202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:33.265202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:33.265202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:33.265202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:33.265202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:33.265202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:33.265202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:33.265202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:33.265202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:33.265202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:33.265202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:33.266650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:26:33.266650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:26:33.266650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:33.266650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:33.266650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 642/1256] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:33.271844 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8434f7-8a04-4acd-a77f-a3cff3ef0371 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:33.366743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:33.380883 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8434f7-8a04-4acd-a77f-a3cff3ef0371 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:33.419364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559365d-0581-4143-93a8-f6d72debee8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:33.420032 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559365d-0581-4143-93a8-f6d72debee8f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:33.420320 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559365d-0581-4143-93a8-f6d72debee8f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:33.420601 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559365d-0581-4143-93a8-f6d72debee8f None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:33.421330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559365d-0581-4143-93a8-f6d72debee8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:33.421636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559365d-0581-4143-93a8-f6d72debee8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:33.423139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559365d-0581-4143-93a8-f6d72debee8f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:33.435058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 615/1257] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:33.440602 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29f524-ff4c-44d6-b0c7-75d2e64afcc2 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:33.548866 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29f524-ff4c-44d6-b0c7-75d2e64afcc2 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:33.585571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70659663-2e6c-4f4a-b29d-2e4734b8cca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:33.586424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70659663-2e6c-4f4a-b29d-2e4734b8cca3 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:33.586706 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70659663-2e6c-4f4a-b29d-2e4734b8cca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:33.587009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70659663-2e6c-4f4a-b29d-2e4734b8cca3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:33.594627 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70659663-2e6c-4f4a-b29d-2e4734b8cca3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:33.596307 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70659663-2e6c-4f4a-b29d-2e4734b8cca3 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:33.611621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-70659663-2e6c-4f4a-b29d-2e4734b8cca3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:26:33.611621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:33.611621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:33.611621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:33.611621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:33.611621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:33.611621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:33.611621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:33.611621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:33.611621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:33.611621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:33.611621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:33.611621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:26:33.611621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:26:33.611621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:26:33.612748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:26:33.612748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:33.612748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:33.612748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:33.612748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:33.612748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:33.612748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:33.612748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:33.612748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:33.612748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:33.612748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:33.612748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:33.612748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:33.612748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:33.612748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:33.612748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:33.612748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:33.614146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:26:33.614146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:26:33.614146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:26:33.614146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:26:33.614146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:26:33.614146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:26:33.614146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:26:33.614146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:33.614146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 643/1258] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:26:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:33.619553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559365d-0581-4143-93a8-f6d72debee8f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:33.730579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559365d-0581-4143-93a8-f6d72debee8f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:33.766197 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb62ed8-d9dc-4269-b757-939dcdcfc3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:33.766883 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb62ed8-d9dc-4269-b757-939dcdcfc3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:33.767201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb62ed8-d9dc-4269-b757-939dcdcfc3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:33.767485 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb62ed8-d9dc-4269-b757-939dcdcfc3a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:33.768294 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb62ed8-d9dc-4269-b757-939dcdcfc3a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:33.768613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb62ed8-d9dc-4269-b757-939dcdcfc3a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:33.770239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb62ed8-d9dc-4269-b757-939dcdcfc3a1 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:33.783685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 616/1259] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:26:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:33.790868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70659663-2e6c-4f4a-b29d-2e4734b8cca3 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:33.819022 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:33.902939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70659663-2e6c-4f4a-b29d-2e4734b8cca3 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:33.939920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf11e7-e407-4659-aa27-7340e8d58be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:33.940604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf11e7-e407-4659-aa27-7340e8d58be0 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:33.940893 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf11e7-e407-4659-aa27-7340e8d58be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:33.941178 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf11e7-e407-4659-aa27-7340e8d58be0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:33.942307 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf11e7-e407-4659-aa27-7340e8d58be0 None admin] RBAC: Adding query filter params (role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.project.id=56da0975019c44409b53be473131143a, user.id=101864afd59543e699189917391e6ab1) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:33.942625 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf11e7-e407-4659-aa27-7340e8d58be0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:33.944131 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf11e7-e407-4659-aa27-7340e8d58be0 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:33.950280 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 644/1260] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:33 2022] GET /identity/v3/role_assignments?user.id=101864afd59543e699189917391e6ab1&scope.project.id=56da0975019c44409b53be473131143a&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:36.029575 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3190aba-e33c-44dc-b5c7-2c751d06199b None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:36.030132 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3190aba-e33c-44dc-b5c7-2c751d06199b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:36.030132 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3190aba-e33c-44dc-b5c7-2c751d06199b None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:36.031555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 617/1261] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:36.036694 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e268746-4a8d-45c2-88bf-748f51530075 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:36.037099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e268746-4a8d-45c2-88bf-748f51530075 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:36.037256 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e268746-4a8d-45c2-88bf-748f51530075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:36.053661 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-1e268746-4a8d-45c2-88bf-748f51530075 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:36.054829 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1e268746-4a8d-45c2-88bf-748f51530075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:36.120718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 645/1262] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:26:36.144046 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3190aba-e33c-44dc-b5c7-2c751d06199b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:36.155907 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:36.244284 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3190aba-e33c-44dc-b5c7-2c751d06199b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:36.277568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7df95-3101-450f-b44d-b673debd437c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:36.278344 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7df95-3101-450f-b44d-b673debd437c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:36.278628 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7df95-3101-450f-b44d-b673debd437c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:36.278906 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7df95-3101-450f-b44d-b673debd437c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:36.288665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7df95-3101-450f-b44d-b673debd437c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:36.297518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-05a7df95-3101-450f-b44d-b673debd437c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:36.297518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:36.297518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:36.297518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:36.297518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:36.297518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:36.297518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:36.297518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:36.297518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:36.297518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:36.297518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:36.297518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:36.297518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:36.297518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:36.297518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:36.297518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:36.298463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:36.298463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:36.298463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:36.298463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:36.298463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:36.298463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:36.298463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:36.298463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:36.298463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:36.298463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:36.298463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:36.298463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:36.298463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:36.298463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:36.298463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:36.298463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:36.299386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:36.299386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:36.299386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:36.299386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:36.299386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:36.299386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:36.299386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:36.299386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:36.299386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:36.299386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:36.299386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:36.299386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:26:36.299386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 618/1263] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:26:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:36.304744 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e268746-4a8d-45c2-88bf-748f51530075 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:36.324678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:36.418166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e268746-4a8d-45c2-88bf-748f51530075 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:36.451924 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb103578-b863-43a7-9609-2e5b3eec32cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:36.452592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb103578-b863-43a7-9609-2e5b3eec32cb None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:36.452875 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb103578-b863-43a7-9609-2e5b3eec32cb None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:36.453152 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb103578-b863-43a7-9609-2e5b3eec32cb None admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:36.453893 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb103578-b863-43a7-9609-2e5b3eec32cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:36.454202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb103578-b863-43a7-9609-2e5b3eec32cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:36.455596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb103578-b863-43a7-9609-2e5b3eec32cb None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:36.464268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 646/1264] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:36.469974 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7df95-3101-450f-b44d-b673debd437c None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:36.538121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:36.572978 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7df95-3101-450f-b44d-b673debd437c None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:36.604998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778ae72-362b-43fd-935d-c124dfe05907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:36.605787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778ae72-362b-43fd-935d-c124dfe05907 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:36.606075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778ae72-362b-43fd-935d-c124dfe05907 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:36.606357 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a778ae72-362b-43fd-935d-c124dfe05907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:36.617204 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a778ae72-362b-43fd-935d-c124dfe05907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:36.618640 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a778ae72-362b-43fd-935d-c124dfe05907 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:36.635831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a778ae72-362b-43fd-935d-c124dfe05907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:36.635831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:36.635831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:36.635831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:36.635831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:36.635831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:36.635831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:36.635831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:36.635831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:36.635831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:36.635831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:36.635831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:36.635831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:26:36.635831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:26:36.635831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:26:36.636864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:26:36.636864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:36.636864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:36.636864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:36.636864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:36.636864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:36.636864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:36.636864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:36.636864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:36.636864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:36.636864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:36.636864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:36.636864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:36.636864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:36.636864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:36.636864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:36.636864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:36.637980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:26:36.637980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:26:36.637980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:36.637980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:26:36.637980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 619/1265] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:36.642817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb103578-b863-43a7-9609-2e5b3eec32cb None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:36.720762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:36.740071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb103578-b863-43a7-9609-2e5b3eec32cb None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:36.772150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f80c68-69ee-4a78-8bc8-ebd777e9edeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:36.772801 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f80c68-69ee-4a78-8bc8-ebd777e9edeb None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:36.773087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f80c68-69ee-4a78-8bc8-ebd777e9edeb None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:36.773367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f80c68-69ee-4a78-8bc8-ebd777e9edeb None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:36.774074 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f80c68-69ee-4a78-8bc8-ebd777e9edeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:36.774384 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f80c68-69ee-4a78-8bc8-ebd777e9edeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:36.775770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f80c68-69ee-4a78-8bc8-ebd777e9edeb None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:36.787371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 647/1266] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:36.792988 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778ae72-362b-43fd-935d-c124dfe05907 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:36.905329 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a778ae72-362b-43fd-935d-c124dfe05907 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:36.930325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:36.945563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b0ffb-3365-4154-b85a-d7d65448b23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:36.946369 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b0ffb-3365-4154-b85a-d7d65448b23e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:36.946658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b0ffb-3365-4154-b85a-d7d65448b23e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:36.946960 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b0ffb-3365-4154-b85a-d7d65448b23e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:36.954468 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b0ffb-3365-4154-b85a-d7d65448b23e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:36.955920 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b0ffb-3365-4154-b85a-d7d65448b23e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:36.970417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-272b0ffb-3365-4154-b85a-d7d65448b23e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:26:36.970417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:36.970417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:36.970417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:36.970417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:36.970417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:36.970417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:36.970417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:36.970417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:36.970417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:36.970417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:36.970417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:36.970417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:26:36.970417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:26:36.970417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:26:36.970417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:26:36.971579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:36.971579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:36.971579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:26:36.971579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:26:36.971579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:26:36.971579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:26:36.971579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:36.971579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:36.971579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:36.971579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:36.971579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:36.971579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:36.971579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:36.971579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:36.971579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:36.971579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:36.971579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:36.971579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:36.972751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:36.972751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:36.972751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:26:36.972751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:26:36.972751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:26:36.972751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:26:36.972751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:26:36.972751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:26:36.972751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:26:36.972751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:26:36.972751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 620/1267] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:36.977611 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f80c68-69ee-4a78-8bc8-ebd777e9edeb None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:37.088912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f80c68-69ee-4a78-8bc8-ebd777e9edeb None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:37.121126 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665105bd-e261-492f-93f3-032fd26e67b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:37.122162 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665105bd-e261-492f-93f3-032fd26e67b6 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:37.122162 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665105bd-e261-492f-93f3-032fd26e67b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:37.122366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665105bd-e261-492f-93f3-032fd26e67b6 None admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:37.123219 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665105bd-e261-492f-93f3-032fd26e67b6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:37.123545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665105bd-e261-492f-93f3-032fd26e67b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:37.124989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665105bd-e261-492f-93f3-032fd26e67b6 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:37.135608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:37.144064 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 648/1268] 10.222.0.43 () {56 vars in 1143 bytes} [Thu Jun 23 22:26:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:37.150179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b0ffb-3365-4154-b85a-d7d65448b23e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:37.249559 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b0ffb-3365-4154-b85a-d7d65448b23e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:37.282779 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8674908-176b-4156-829d-5db6f4a2264e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:37.283590 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8674908-176b-4156-829d-5db6f4a2264e None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:37.283882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8674908-176b-4156-829d-5db6f4a2264e None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:37.284167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8674908-176b-4156-829d-5db6f4a2264e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:37.295038 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8674908-176b-4156-829d-5db6f4a2264e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:37.296481 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8674908-176b-4156-829d-5db6f4a2264e None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:37.313157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a8674908-176b-4156-829d-5db6f4a2264e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:37.313157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:37.313157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:37.313157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:37.313157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:37.313157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:37.313157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:37.313157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:37.313157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:37.313157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:37.313157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:37.313157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:37.313157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:26:37.313157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:26:37.313157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:26:37.314207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:26:37.314207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:37.314207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:37.314207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:37.314207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:37.314207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:37.314207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:37.314207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:37.314207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:37.314207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:37.314207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:37.314207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:37.314207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:37.314207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:37.314207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:37.314207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:37.314207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:37.315319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:26:37.315319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:26:37.315319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:37.315319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:26:37.315319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 621/1269] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:37.320374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665105bd-e261-492f-93f3-032fd26e67b6 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:37.417546 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665105bd-e261-492f-93f3-032fd26e67b6 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:37.449736 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab4a76-c476-44cc-a77d-bed190e964e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:37.450396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab4a76-c476-44cc-a77d-bed190e964e2 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:37.450687 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab4a76-c476-44cc-a77d-bed190e964e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:37.450994 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab4a76-c476-44cc-a77d-bed190e964e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:37.451709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab4a76-c476-44cc-a77d-bed190e964e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:37.452021 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab4a76-c476-44cc-a77d-bed190e964e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:37.453382 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab4a76-c476-44cc-a77d-bed190e964e2 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:37.464714 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 649/1270] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:37.470318 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8674908-176b-4156-829d-5db6f4a2264e None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:37.501670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:37.569242 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8674908-176b-4156-829d-5db6f4a2264e None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:37.602037 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464c86e-5863-4e8f-adb9-ca162895dab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:37.602910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464c86e-5863-4e8f-adb9-ca162895dab0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:37.603234 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464c86e-5863-4e8f-adb9-ca162895dab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:37.603519 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464c86e-5863-4e8f-adb9-ca162895dab0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:37.609958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4464c86e-5863-4e8f-adb9-ca162895dab0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:37.611439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4464c86e-5863-4e8f-adb9-ca162895dab0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:37.625581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-4464c86e-5863-4e8f-adb9-ca162895dab0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:26:37.625581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:37.625581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:37.625581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:37.625581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:37.625581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:37.625581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:37.625581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:37.625581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:37.625581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:37.625581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:37.625581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:37.625581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:26:37.625581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:26:37.625581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:26:37.626643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:26:37.626643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:37.626643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:37.626643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:37.626643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:37.626643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:37.626643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:37.626643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:37.626643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:37.626643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:37.626643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:37.626643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:37.626643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:37.626643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:37.626643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:37.626643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:37.626643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:37.627763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:26:37.627763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:26:37.627763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:26:37.627763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:26:37.627763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:26:37.627763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:26:37.627763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:26:37.627763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:26:37.627763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 622/1271] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:26:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:37.633446 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab4a76-c476-44cc-a77d-bed190e964e2 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:37.689587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:37.730893 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab4a76-c476-44cc-a77d-bed190e964e2 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:37.763911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49de86-ac51-432d-a822-248fb193b594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:37.764501 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49de86-ac51-432d-a822-248fb193b594 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:37.764785 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49de86-ac51-432d-a822-248fb193b594 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:37.765067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49de86-ac51-432d-a822-248fb193b594 None admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:37.765858 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f49de86-ac51-432d-a822-248fb193b594 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:37.766153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f49de86-ac51-432d-a822-248fb193b594 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:37.767601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f49de86-ac51-432d-a822-248fb193b594 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:37.779955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 650/1272] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:26:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:37.785797 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464c86e-5863-4e8f-adb9-ca162895dab0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:37.877178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:37.886637 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464c86e-5863-4e8f-adb9-ca162895dab0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:37.919976 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94dccd-e3c2-4abe-907e-d40f6c65e2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:37.920854 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94dccd-e3c2-4abe-907e-d40f6c65e2f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:37.921139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94dccd-e3c2-4abe-907e-d40f6c65e2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:37.921422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94dccd-e3c2-4abe-907e-d40f6c65e2f3 None admin] PATH_INFO: `/v3/projects/56da0975019c44409b53be473131143a/users/101864afd59543e699189917391e6ab1/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:37.923356 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94dccd-e3c2-4abe-907e-d40f6c65e2f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=56da0975019c44409b53be473131143a, user_id=101864afd59543e699189917391e6ab1, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:37.924768 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94dccd-e3c2-4abe-907e-d40f6c65e2f3 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:37.936850 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 623/1273] 10.222.0.43 () {58 vars in 1439 bytes} [Thu Jun 23 22:26:37 2022] PUT /identity/v3/projects/56da0975019c44409b53be473131143a/users/101864afd59543e699189917391e6ab1/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:26:39.932496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a403f6b-eda9-4f63-9b88-f72cbd234145 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:39.933025 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a403f6b-eda9-4f63-9b88-f72cbd234145 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:39.933135 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a403f6b-eda9-4f63-9b88-f72cbd234145 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:39.934422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 651/1274] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:39.939698 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58b70b-f107-4bda-a72d-75f7a9ce0f88 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:39.940118 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58b70b-f107-4bda-a72d-75f7a9ce0f88 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:39.940261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58b70b-f107-4bda-a72d-75f7a9ce0f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:39.957038 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-af58b70b-f107-4bda-a72d-75f7a9ce0f88 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:39.958838 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-af58b70b-f107-4bda-a72d-75f7a9ce0f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:40.023560 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 624/1275] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:26:40.047967 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a403f6b-eda9-4f63-9b88-f72cbd234145 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:40.147209 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a403f6b-eda9-4f63-9b88-f72cbd234145 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:40.164933 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:40.182830 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13f2ad-820d-4750-9059-d24110ee349a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:40.183582 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13f2ad-820d-4750-9059-d24110ee349a None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:40.183868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13f2ad-820d-4750-9059-d24110ee349a None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:40.184145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13f2ad-820d-4750-9059-d24110ee349a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:40.194607 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13f2ad-820d-4750-9059-d24110ee349a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 22:26:40.202091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-ee13f2ad-820d-4750-9059-d24110ee349a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:40.202091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:40.202091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:40.202091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:40.202091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:40.202091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:40.202091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:40.202091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:40.202091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:40.202091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:40.202091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:40.202091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:40.202091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 22:26:40.202091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 22:26:40.202091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 22:26:40.202091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 22:26:40.203236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 22:26:40.203236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 22:26:40.203236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 22:26:40.203236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 22:26:40.203236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:40.203236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:40.203236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:40.203236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:40.203236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:40.203236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:40.203236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:40.203236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:40.203236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:40.203236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:40.203236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:40.203236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:40.204291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:40.204291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:40.204291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:40.204291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:40.204291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 22:26:40.204291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 22:26:40.204291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 22:26:40.204291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 22:26:40.204291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 22:26:40.204291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 22:26:40.204291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 22:26:40.204291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:40.204291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 652/1276] 10.222.0.43 () {56 vars in 1096 bytes} [Thu Jun 23 22:26:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:40.209751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58b70b-f107-4bda-a72d-75f7a9ce0f88 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:40.288345 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:40.306824 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58b70b-f107-4bda-a72d-75f7a9ce0f88 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:40.339955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a12e55-6d1c-4fd9-9bce-35b7eb9c06f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:40.340613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a12e55-6d1c-4fd9-9bce-35b7eb9c06f7 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:40.340903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a12e55-6d1c-4fd9-9bce-35b7eb9c06f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:40.341185 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a12e55-6d1c-4fd9-9bce-35b7eb9c06f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:40.341927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a12e55-6d1c-4fd9-9bce-35b7eb9c06f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:40.342241 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a12e55-6d1c-4fd9-9bce-35b7eb9c06f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:40.343627 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a12e55-6d1c-4fd9-9bce-35b7eb9c06f7 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:40.352118 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 625/1277] 10.222.0.43 () {56 vars in 1099 bytes} [Thu Jun 23 22:26:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:40.357643 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13f2ad-820d-4750-9059-d24110ee349a None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:40.458948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13f2ad-820d-4750-9059-d24110ee349a None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:40.491769 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9df4a-2c8a-4f45-948c-2382d0699aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:40.492562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9df4a-2c8a-4f45-948c-2382d0699aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:40.492852 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9df4a-2c8a-4f45-948c-2382d0699aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:40.493137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9df4a-2c8a-4f45-948c-2382d0699aa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:40.504175 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9df4a-2c8a-4f45-948c-2382d0699aa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:40.505718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9df4a-2c8a-4f45-948c-2382d0699aa6 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:40.522988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-edb9df4a-2c8a-4f45-948c-2382d0699aa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:40.522988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:40.522988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:40.522988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:40.522988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:40.522988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:40.522988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:40.522988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:40.522988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:40.522988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:40.522988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:40.522988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:40.522988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:26:40.522988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:26:40.522988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:26:40.524141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:26:40.524141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:40.524141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:40.524141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:40.524141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:40.524141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:40.524141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:40.524141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:40.524141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:40.524141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:40.524141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:40.524141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:40.524141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:40.524141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:40.524141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:40.524141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:40.524141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:40.525433 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:26:40.525433 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:26:40.525433 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:40.525433 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:40.525433 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 653/1278] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:40.530068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a12e55-6d1c-4fd9-9bce-35b7eb9c06f7 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:40.625434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a12e55-6d1c-4fd9-9bce-35b7eb9c06f7 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:40.658055 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e911d3-38b2-4dc1-8b8a-144f4e7256de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:40.658696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e911d3-38b2-4dc1-8b8a-144f4e7256de None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:40.659010 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e911d3-38b2-4dc1-8b8a-144f4e7256de None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:40.659294 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e911d3-38b2-4dc1-8b8a-144f4e7256de None admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:40.660004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e911d3-38b2-4dc1-8b8a-144f4e7256de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:40.660348 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e911d3-38b2-4dc1-8b8a-144f4e7256de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:40.662183 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e911d3-38b2-4dc1-8b8a-144f4e7256de None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:40.673645 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 626/1279] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:40.679183 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9df4a-2c8a-4f45-948c-2382d0699aa6 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:40.713486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:40.783605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9df4a-2c8a-4f45-948c-2382d0699aa6 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:40.815487 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2fdc9f-8300-4d4d-a972-f0fa7f563738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:40.816297 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fdc9f-8300-4d4d-a972-f0fa7f563738 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:40.817111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fdc9f-8300-4d4d-a972-f0fa7f563738 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:40.817393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fdc9f-8300-4d4d-a972-f0fa7f563738 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:40.824737 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2fdc9f-8300-4d4d-a972-f0fa7f563738 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:40.826177 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2fdc9f-8300-4d4d-a972-f0fa7f563738 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:40.840745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-0b2fdc9f-8300-4d4d-a972-f0fa7f563738 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:26:40.840745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:40.840745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:40.840745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:40.840745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:40.840745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:40.840745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:40.840745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:40.840745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:40.840745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:40.840745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:40.840745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:40.840745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 22:26:40.840745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 22:26:40.840745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 22:26:40.840745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 22:26:40.841839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:40.841839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:40.841839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 22:26:40.841839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:26:40.841839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 22:26:40.841839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 22:26:40.841839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:40.841839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:40.841839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:40.841839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:40.841839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:40.841839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:40.841839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:40.841839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:40.841839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:40.841839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:40.841839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:40.841839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:40.843099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:40.843099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:40.843099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 22:26:40.843099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 22:26:40.843099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 22:26:40.843099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 22:26:40.843099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 22:26:40.843099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 22:26:40.843099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 22:26:40.843099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:40.843099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 654/1280] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:40.848098 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e911d3-38b2-4dc1-8b8a-144f4e7256de None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:40.852598 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:40.959282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e911d3-38b2-4dc1-8b8a-144f4e7256de None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:40.993729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cc44d-603d-4f80-8d8b-7a9cf2666a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:40.994395 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cc44d-603d-4f80-8d8b-7a9cf2666a9f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:40.994682 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cc44d-603d-4f80-8d8b-7a9cf2666a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:40.994992 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cc44d-603d-4f80-8d8b-7a9cf2666a9f None admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:40.995824 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cc44d-603d-4f80-8d8b-7a9cf2666a9f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:40.996140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cc44d-603d-4f80-8d8b-7a9cf2666a9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:40.997553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cc44d-603d-4f80-8d8b-7a9cf2666a9f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:41.015089 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 627/1281] 10.222.0.43 () {56 vars in 1143 bytes} [Thu Jun 23 22:26:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:41.021275 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2fdc9f-8300-4d4d-a972-f0fa7f563738 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:41.109384 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:41.119096 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2fdc9f-8300-4d4d-a972-f0fa7f563738 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:41.152565 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bce7f5-fe35-4c8e-93ad-77978326591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:41.153208 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bce7f5-fe35-4c8e-93ad-77978326591c None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:41.153491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bce7f5-fe35-4c8e-93ad-77978326591c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:41.153769 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bce7f5-fe35-4c8e-93ad-77978326591c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:41.219996 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bce7f5-fe35-4c8e-93ad-77978326591c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:41.221447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bce7f5-fe35-4c8e-93ad-77978326591c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:41.309260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:41.319935 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 655/1282] 10.222.0.43 () {58 vars in 1282 bytes} [Thu Jun 23 22:26:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 22:26:41.325710 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cc44d-603d-4f80-8d8b-7a9cf2666a9f None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:41.385614 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:41.422641 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cc44d-603d-4f80-8d8b-7a9cf2666a9f None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:41.455357 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c68a8e-59db-41d6-9ec8-effec926f2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:41.456140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c68a8e-59db-41d6-9ec8-effec926f2cc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:41.456427 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c68a8e-59db-41d6-9ec8-effec926f2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:41.456711 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c68a8e-59db-41d6-9ec8-effec926f2cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:41.466914 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c68a8e-59db-41d6-9ec8-effec926f2cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:41.468375 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c68a8e-59db-41d6-9ec8-effec926f2cc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:41.485290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-d2c68a8e-59db-41d6-9ec8-effec926f2cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:41.485290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:41.485290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:41.485290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:41.485290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:41.485290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:41.485290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:41.485290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:41.485290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:41.485290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:41.485290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:41.485290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:41.485290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 22:26:41.485290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 22:26:41.485290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 22:26:41.486327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 22:26:41.486327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:41.486327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:41.486327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:41.486327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:41.486327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:41.486327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:41.486327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:41.486327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:41.486327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:41.486327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:41.486327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:41.486327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:41.486327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:41.486327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:41.486327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:41.486327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:41.487412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 22:26:41.487412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 22:26:41.487412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 22:26:41.487412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:26:41.487412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 628/1283] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:26:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:41.492424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bce7f5-fe35-4c8e-93ad-77978326591c None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:41.589546 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bce7f5-fe35-4c8e-93ad-77978326591c None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:41.622682 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec17a94-1b40-4361-8b90-643599c049cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:41.623374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec17a94-1b40-4361-8b90-643599c049cf None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:41.623663 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec17a94-1b40-4361-8b90-643599c049cf None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:41.623940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec17a94-1b40-4361-8b90-643599c049cf None admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:41.624654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec17a94-1b40-4361-8b90-643599c049cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:41.624963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec17a94-1b40-4361-8b90-643599c049cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:41.626322 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec17a94-1b40-4361-8b90-643599c049cf None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:41.638059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 656/1284] 10.222.0.43 () {56 vars in 1109 bytes} [Thu Jun 23 22:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:41.643552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c68a8e-59db-41d6-9ec8-effec926f2cc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:41.740749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c68a8e-59db-41d6-9ec8-effec926f2cc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:41.753895 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:41.774022 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40907c7-b007-4716-a681-b05bf7454cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:41.774796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40907c7-b007-4716-a681-b05bf7454cdc None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:41.775108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40907c7-b007-4716-a681-b05bf7454cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:41.775390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40907c7-b007-4716-a681-b05bf7454cdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:41.781982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40907c7-b007-4716-a681-b05bf7454cdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:41.783431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40907c7-b007-4716-a681-b05bf7454cdc None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:41.796550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application [None req-a40907c7-b007-4716-a681-b05bf7454cdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:26:41.796550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:41.796550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:41.796550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:41.796550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:41.796550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:41.796550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:41.796550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:41.796550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:41.796550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:41.796550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:41.796550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:41.796550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 22:26:41.796550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 22:26:41.796550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 22:26:41.797581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 22:26:41.797581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:41.797581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:41.797581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:41.797581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:41.797581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:41.797581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:41.797581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:41.797581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:41.797581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:41.797581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:41.797581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:41.797581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:41.797581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:41.797581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:41.797581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:41.797581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:41.798680 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 22:26:41.798680 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 22:26:41.798680 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 22:26:41.798680 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 22:26:41.798680 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 22:26:41.798680 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 22:26:41.798680 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 22:26:41.798680 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: ERROR keystone.server.flask.application Jun 23 22:26:41.798680 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 629/1285] 10.222.0.43 () {56 vars in 1111 bytes} [Thu Jun 23 22:26:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:41.803985 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec17a94-1b40-4361-8b90-643599c049cf None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:41.852578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:41.902260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec17a94-1b40-4361-8b90-643599c049cf None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:41.934772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d2d6d-02ad-43af-9038-42fa199d931a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:41.935459 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d2d6d-02ad-43af-9038-42fa199d931a None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:41.935750 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d2d6d-02ad-43af-9038-42fa199d931a None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:41.936034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d2d6d-02ad-43af-9038-42fa199d931a None admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:41.936821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d2d6d-02ad-43af-9038-42fa199d931a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:41.937137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d2d6d-02ad-43af-9038-42fa199d931a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:41.938565 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d2d6d-02ad-43af-9038-42fa199d931a None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:41.950468 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 657/1286] 10.222.0.43 () {56 vars in 1148 bytes} [Thu Jun 23 22:26:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:26:41.956300 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40907c7-b007-4716-a681-b05bf7454cdc None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:42.052704 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40907c7-b007-4716-a681-b05bf7454cdc None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:42.084481 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cba431-0de7-4d77-b0d6-f9077e1c512f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:42.085116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cba431-0de7-4d77-b0d6-f9077e1c512f None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:42.085398 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cba431-0de7-4d77-b0d6-f9077e1c512f None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:42.085682 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cba431-0de7-4d77-b0d6-f9077e1c512f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:42.086501 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cba431-0de7-4d77-b0d6-f9077e1c512f None admin] RBAC: Adding query filter params (role.id=950c82bd56ab4b4f89ece13a8e6e61f2, scope.project.id=56da0975019c44409b53be473131143a, user.id=101864afd59543e699189917391e6ab1) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:42.086814 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cba431-0de7-4d77-b0d6-f9077e1c512f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:42.088251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cba431-0de7-4d77-b0d6-f9077e1c512f None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:42.093833 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 630/1287] 10.222.0.43 () {56 vars in 1374 bytes} [Thu Jun 23 22:26:41 2022] GET /identity/v3/role_assignments?user.id=101864afd59543e699189917391e6ab1&scope.project.id=56da0975019c44409b53be473131143a&role.id=950c82bd56ab4b4f89ece13a8e6e61f2 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:44.148100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693d982-19a9-4677-9019-1a2f4c789cf5 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:44.148571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693d982-19a9-4677-9019-1a2f4c789cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:44.148665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693d982-19a9-4677-9019-1a2f4c789cf5 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:44.150023 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 658/1288] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:44.155464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826cecb-bcf5-44ef-86b7-a87db34f52d9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:44.155877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826cecb-bcf5-44ef-86b7-a87db34f52d9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:44.156032 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826cecb-bcf5-44ef-86b7-a87db34f52d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:44.173447 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f826cecb-bcf5-44ef-86b7-a87db34f52d9 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:44.174623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f826cecb-bcf5-44ef-86b7-a87db34f52d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:44.204062 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:44.241135 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 631/1289] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:26:44.263335 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693d982-19a9-4677-9019-1a2f4c789cf5 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:44.364001 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693d982-19a9-4677-9019-1a2f4c789cf5 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:44.374175 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:44.400570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953132df-5ca7-4a1d-9c0e-c433b568bfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:44.401553 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953132df-5ca7-4a1d-9c0e-c433b568bfbc None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:44.401837 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953132df-5ca7-4a1d-9c0e-c433b568bfbc None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:44.402113 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953132df-5ca7-4a1d-9c0e-c433b568bfbc None admin] PATH_INFO: `/v3/services/placement` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:44.402761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953132df-5ca7-4a1d-9c0e-c433b568bfbc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:44.404153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953132df-5ca7-4a1d-9c0e-c433b568bfbc None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:44.416063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-953132df-5ca7-4a1d-9c0e-c433b568bfbc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 22:26:44.416063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:44.416063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:26:44.416063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:26:44.416063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:26:44.416063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:26:44.416063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:26:44.416063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:26:44.416063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 22:26:44.416063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:44.416063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:26:44.416063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:44.416063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:44.416063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:44.416063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:44.416063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:44.416063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:44.417301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:44.417301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:44.417301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:44.417301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:44.417301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:44.417301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:44.417301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:26:44.417301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:26:44.417301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:26:44.417301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:26:44.417301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:44.417301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:44.417301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:44.417301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:44.417301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:44.417301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:44.418441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:44.418441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:44.418441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:44.418441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:44.418441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:44.418441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:44.418441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:44.418441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:44.418441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:44.418441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:44.418441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:26:44.418441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:26:44.418441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 22:26:44.418441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:44.418441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 659/1290] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 22:26:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:44.423272 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826cecb-bcf5-44ef-86b7-a87db34f52d9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:44.522789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826cecb-bcf5-44ef-86b7-a87db34f52d9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:44.561787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807ad22-838d-4480-ac2b-3badccdd170c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:44.562906 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807ad22-838d-4480-ac2b-3badccdd170c None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:44.563226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807ad22-838d-4480-ac2b-3badccdd170c None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:44.563506 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807ad22-838d-4480-ac2b-3badccdd170c None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:44.564337 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807ad22-838d-4480-ac2b-3badccdd170c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:44.564895 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807ad22-838d-4480-ac2b-3badccdd170c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:44.567343 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807ad22-838d-4480-ac2b-3badccdd170c None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:44.575416 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 632/1291] 10.222.0.43 () {56 vars in 1116 bytes} [Thu Jun 23 22:26:44 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:44.581117 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953132df-5ca7-4a1d-9c0e-c433b568bfbc None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:44.700958 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953132df-5ca7-4a1d-9c0e-c433b568bfbc None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:44.746588 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b5e3e-b35b-4494-8527-f7b482d3e650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:44.747780 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b5e3e-b35b-4494-8527-f7b482d3e650 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:44.748317 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b5e3e-b35b-4494-8527-f7b482d3e650 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:44.748727 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b5e3e-b35b-4494-8527-f7b482d3e650 None admin] PATH_INFO: `/v3/services` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:44.749442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b5e3e-b35b-4494-8527-f7b482d3e650 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:44.749747 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b5e3e-b35b-4494-8527-f7b482d3e650 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:44.751133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b5e3e-b35b-4494-8527-f7b482d3e650 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:44.756326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 660/1292] 10.222.0.43 () {56 vars in 1116 bytes} [Thu Jun 23 22:26:44 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:46.863299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88047c38-4f2b-439f-96ed-bacb50484758 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:46.864433 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88047c38-4f2b-439f-96ed-bacb50484758 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:46.864433 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88047c38-4f2b-439f-96ed-bacb50484758 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:46.865975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 633/1293] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:26:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:46.871399 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbe86d-550d-45d4-a0ec-6adf00016013 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:46.871925 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbe86d-550d-45d4-a0ec-6adf00016013 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:46.871925 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbe86d-550d-45d4-a0ec-6adf00016013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:46.889043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-19cbe86d-550d-45d4-a0ec-6adf00016013 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:46.890356 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-19cbe86d-550d-45d4-a0ec-6adf00016013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:46.922047 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:46.959759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 661/1294] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 22:26:46.983198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88047c38-4f2b-439f-96ed-bacb50484758 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:47.014402 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:47.085657 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88047c38-4f2b-439f-96ed-bacb50484758 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:47.119261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec78a9e8-f493-4f47-995e-1730768d4356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:47.119852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78a9e8-f493-4f47-995e-1730768d4356 None admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:47.120142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78a9e8-f493-4f47-995e-1730768d4356 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:47.120424 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78a9e8-f493-4f47-995e-1730768d4356 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:47.121150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78a9e8-f493-4f47-995e-1730768d4356 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:47.122527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec78a9e8-f493-4f47-995e-1730768d4356 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:47.136664 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 634/1295] 10.222.0.43 () {60 vars in 1141 bytes} [Thu Jun 23 22:26:46 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:26:49.176961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e6105-8c2a-4ed1-b6e3-36021c435004 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:49.177448 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e6105-8c2a-4ed1-b6e3-36021c435004 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:49.177557 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e6105-8c2a-4ed1-b6e3-36021c435004 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:49.178892 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 662/1296] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:49.183925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb65bc-7420-4581-848a-abef668891dd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:49.184293 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb65bc-7420-4581-848a-abef668891dd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:49.184498 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb65bc-7420-4581-848a-abef668891dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:49.201388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fbdb65bc-7420-4581-848a-abef668891dd None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:49.202572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fbdb65bc-7420-4581-848a-abef668891dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:49.273189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 635/1297] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:49 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:26:49.295993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e6105-8c2a-4ed1-b6e3-36021c435004 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:49.395924 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e6105-8c2a-4ed1-b6e3-36021c435004 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:49.429086 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3d93f-7467-487b-ad21-9104682af31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:49.429878 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3d93f-7467-487b-ad21-9104682af31e None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:49.430167 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3d93f-7467-487b-ad21-9104682af31e None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:49.430450 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3d93f-7467-487b-ad21-9104682af31e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:49.431134 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3d93f-7467-487b-ad21-9104682af31e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:49.432510 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3d93f-7467-487b-ad21-9104682af31e None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:49.444853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-c0d3d93f-7467-487b-ad21-9104682af31e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 22:26:49.444853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:49.444853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:26:49.444853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:26:49.444853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:26:49.444853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:26:49.444853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:26:49.444853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:26:49.444853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 22:26:49.444853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:49.444853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:26:49.444853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:49.444853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:49.444853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:49.444853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:49.444853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:49.444853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:49.446030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:49.446030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:49.446030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:49.446030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:49.446030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:49.446030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:49.446030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:26:49.446030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:26:49.446030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:26:49.446030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:26:49.446030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:49.446030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:49.446030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:49.446030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:49.446030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:49.446030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:49.446890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:49.446890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:49.446890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:49.446890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:49.446890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:49.446890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:49.446890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:49.446890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:49.446890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:49.446890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:49.446890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:26:49.446890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:26:49.446890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 22:26:49.446890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:49.446890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 663/1298] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 22:26:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:49.451982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb65bc-7420-4581-848a-abef668891dd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:49.469733 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:49.550703 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb65bc-7420-4581-848a-abef668891dd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:49.584925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4700b9-fd94-499d-9587-c13b7fc085c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:49.585586 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4700b9-fd94-499d-9587-c13b7fc085c0 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:49.585878 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4700b9-fd94-499d-9587-c13b7fc085c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:49.586166 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4700b9-fd94-499d-9587-c13b7fc085c0 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:49.586873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4700b9-fd94-499d-9587-c13b7fc085c0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:49.587219 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4700b9-fd94-499d-9587-c13b7fc085c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:49.588606 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4700b9-fd94-499d-9587-c13b7fc085c0 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:49.593675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 636/1299] 10.222.0.43 () {56 vars in 1116 bytes} [Thu Jun 23 22:26:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:49.600290 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3d93f-7467-487b-ad21-9104682af31e None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:49.631626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:49.698791 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3d93f-7467-487b-ad21-9104682af31e None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:49.731200 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480b516-26cb-42d2-bf6d-e526d6fdf636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:49.731850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480b516-26cb-42d2-bf6d-e526d6fdf636 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:49.732150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480b516-26cb-42d2-bf6d-e526d6fdf636 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:49.732437 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480b516-26cb-42d2-bf6d-e526d6fdf636 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:49.733227 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480b516-26cb-42d2-bf6d-e526d6fdf636 None admin] RBAC: Adding query filter params (interface=public, service_id=cd52168054a24fd2bd354b7584532df5, region_id=RegionOne) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:49.733549 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480b516-26cb-42d2-bf6d-e526d6fdf636 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:49.734912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480b516-26cb-42d2-bf6d-e526d6fdf636 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:49.740310 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 664/1300] 10.222.0.43 () {56 vars in 1251 bytes} [Thu Jun 23 22:26:49 2022] GET /identity/v3/endpoints?service_id=cd52168054a24fd2bd354b7584532df5&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:49.745875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4700b9-fd94-499d-9587-c13b7fc085c0 None admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:49.825530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:49.844102 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4700b9-fd94-499d-9587-c13b7fc085c0 None admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:49.881193 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f6d2f-0928-4b51-b128-a6270b2bff06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:49.881854 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f6d2f-0928-4b51-b128-a6270b2bff06 None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:49.882147 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f6d2f-0928-4b51-b128-a6270b2bff06 None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:49.882434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f6d2f-0928-4b51-b128-a6270b2bff06 None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:49.883147 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f6d2f-0928-4b51-b128-a6270b2bff06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:49.884505 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f6d2f-0928-4b51-b128-a6270b2bff06 None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:49.891255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 637/1301] 10.222.0.43 () {56 vars in 1087 bytes} [Thu Jun 23 22:26:49 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:26:51.899538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc09689-2176-4d7a-a651-ea642d776fad None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:51.900065 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc09689-2176-4d7a-a651-ea642d776fad None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:51.900190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc09689-2176-4d7a-a651-ea642d776fad None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:51.901713 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 665/1302] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:26:51.906770 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69776ad6-a582-4769-a636-98205188afaf None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:51.907239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69776ad6-a582-4769-a636-98205188afaf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:51.907370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69776ad6-a582-4769-a636-98205188afaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:51.923681 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-69776ad6-a582-4769-a636-98205188afaf None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:26:51.924856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-69776ad6-a582-4769-a636-98205188afaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:26:51.989930 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 638/1303] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:26:51 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 22:26:52.013080 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc09689-2176-4d7a-a651-ea642d776fad None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:52.107597 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:52.114446 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc09689-2176-4d7a-a651-ea642d776fad None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:52.147957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba6020-6204-4a66-a965-c36cc39d9c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:52.148746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba6020-6204-4a66-a965-c36cc39d9c22 None admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:52.149031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba6020-6204-4a66-a965-c36cc39d9c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:52.149306 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba6020-6204-4a66-a965-c36cc39d9c22 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:52.149995 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba6020-6204-4a66-a965-c36cc39d9c22 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:52.151594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba6020-6204-4a66-a965-c36cc39d9c22 None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:52.163447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application [None req-4cba6020-6204-4a66-a965-c36cc39d9c22 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 22:26:52.163447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:52.163447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 22:26:52.163447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 22:26:52.163447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 22:26:52.163447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 22:26:52.163447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 22:26:52.163447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:26:52.163447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 22:26:52.163447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:52.163447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 22:26:52.163447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:52.163447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 22:26:52.163447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 22:26:52.163447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 22:26:52.163447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 22:26:52.163447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 22:26:52.164504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 22:26:52.164504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 22:26:52.164504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 22:26:52.164504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 22:26:52.164504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 22:26:52.164504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 22:26:52.164504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 22:26:52.164504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 22:26:52.164504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 22:26:52.164504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 22:26:52.164504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 22:26:52.164504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 22:26:52.164504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 22:26:52.164504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 22:26:52.164504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 22:26:52.164504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 22:26:52.165418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 22:26:52.165418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application with Lock( Jun 23 22:26:52.165418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 22:26:52.165418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self._enter() Jun 23 22:26:52.165418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 22:26:52.165418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 22:26:52.165418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 22:26:52.165418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application return self.creator() Jun 23 22:26:52.165418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 22:26:52.165418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application created_value = creator( Jun 23 22:26:52.165418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 22:26:52.165418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 22:26:52.165418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 22:26:52.165418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: ERROR keystone.server.flask.application Jun 23 22:26:52.165418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 666/1304] 10.222.0.43 () {56 vars in 1117 bytes} [Thu Jun 23 22:26:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 22:26:52.170803 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69776ad6-a582-4769-a636-98205188afaf None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:52.243032 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:26:52.265462 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69776ad6-a582-4769-a636-98205188afaf None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:52.297349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff3916-bfe4-46d1-9724-d8e206d234aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:52.298002 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff3916-bfe4-46d1-9724-d8e206d234aa None admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:52.298289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff3916-bfe4-46d1-9724-d8e206d234aa None admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:52.298572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff3916-bfe4-46d1-9724-d8e206d234aa None admin] PATH_INFO: `/v3/services` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:52.299299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff3916-bfe4-46d1-9724-d8e206d234aa None admin] RBAC: Adding query filter params (name=placement) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:26:52.299611 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff3916-bfe4-46d1-9724-d8e206d234aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:52.301605 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff3916-bfe4-46d1-9724-d8e206d234aa None admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:52.306767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 639/1305] 10.222.0.43 () {56 vars in 1116 bytes} [Thu Jun 23 22:26:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:26:52.312479 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba6020-6204-4a66-a965-c36cc39d9c22 None admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:26:52.410898 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba6020-6204-4a66-a965-c36cc39d9c22 None admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:26:52.444152 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb26bd3-e7cd-41a6-9670-11ea8178b86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:26:52.444800 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb26bd3-e7cd-41a6-9670-11ea8178b86c None admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:26:52.445087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb26bd3-e7cd-41a6-9670-11ea8178b86c None admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:26:52.445371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb26bd3-e7cd-41a6-9670-11ea8178b86c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:26:52.446101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb26bd3-e7cd-41a6-9670-11ea8178b86c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:26:52.447500 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb26bd3-e7cd-41a6-9670-11ea8178b86c None admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:26:52.464501 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 667/1306] 10.222.0.43 () {60 vars in 1144 bytes} [Thu Jun 23 22:26:52 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:27:02.303827 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=30 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:02.460376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:55.511280 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdc8c3-c6ca-40c0-928e-131668cab1a5 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:55.511860 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdc8c3-c6ca-40c0-928e-131668cab1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:55.511860 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdc8c3-c6ca-40c0-928e-131668cab1a5 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:55.517667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 640/1307] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:27:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:55.518965 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1dc158-4a1e-4f29-8bee-0925727288e2 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:55.519941 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1dc158-4a1e-4f29-8bee-0925727288e2 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:55.520083 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1dc158-4a1e-4f29-8bee-0925727288e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:55.593311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-3b1dc158-4a1e-4f29-8bee-0925727288e2 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:55.594543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3b1dc158-4a1e-4f29-8bee-0925727288e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:55.787512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 668/1308] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:27:55.836486 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17143a0-4c6e-4971-bdef-0879a973b73f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:55.836696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17143a0-4c6e-4971-bdef-0879a973b73f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:55.836984 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17143a0-4c6e-4971-bdef-0879a973b73f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:55.838900 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 641/1309] 10.222.0.43 () {54 vars in 964 bytes} [Thu Jun 23 22:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:55.846565 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cd383-9002-4675-b5f1-a2ea14bd5b6d None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:55.848109 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cd383-9002-4675-b5f1-a2ea14bd5b6d None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:55.849005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cd383-9002-4675-b5f1-a2ea14bd5b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:55.944781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-8e8cd383-9002-4675-b5f1-a2ea14bd5b6d None None] MFA Rules not processed for user `37ba527b96634fc2a48b6ca9597d8191`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:55.945980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8e8cd383-9002-4675-b5f1-a2ea14bd5b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:55.967031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:56.030517 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 669/1310] 10.222.0.43 () {58 vars in 1061 bytes} [Thu Jun 23 22:27:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:27:56.039801 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17143a0-4c6e-4971-bdef-0879a973b73f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:56.041612 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c17143a0-4c6e-4971-bdef-0879a973b73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:56.165474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17143a0-4c6e-4971-bdef-0879a973b73f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:56.250478 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd2a59-1da6-4937-9d93-db823818d8de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:56.251161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd2a59-1da6-4937-9d93-db823818d8de service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:56.251445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd2a59-1da6-4937-9d93-db823818d8de service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:56.251720 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd2a59-1da6-4937-9d93-db823818d8de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:56.253361 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fbfd2a59-1da6-4937-9d93-db823818d8de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:56.314165 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:56.384589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd2a59-1da6-4937-9d93-db823818d8de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:56.386159 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd2a59-1da6-4937-9d93-db823818d8de service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:56.407591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 642/1311] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:27:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:27:58.742179 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b51d1f-0914-49a3-94cc-d11d8bb0e0fc None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:58.742721 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b51d1f-0914-49a3-94cc-d11d8bb0e0fc None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:58.742843 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b51d1f-0914-49a3-94cc-d11d8bb0e0fc None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:58.744191 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 670/1312] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:58.749729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc8b4d-cf94-449d-ba16-224d28be4764 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:58.750223 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc8b4d-cf94-449d-ba16-224d28be4764 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:58.750314 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc8b4d-cf94-449d-ba16-224d28be4764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:58.831014 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b1cc8b4d-cf94-449d-ba16-224d28be4764 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:58.832298 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b1cc8b4d-cf94-449d-ba16-224d28be4764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:58.930781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 643/1313] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:27:58.983492 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae62362-d967-4b86-b583-7328111c952b None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:58.983785 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae62362-d967-4b86-b583-7328111c952b None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:58.984075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae62362-d967-4b86-b583-7328111c952b None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:58.985485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 671/1314] 10.222.0.43 () {54 vars in 964 bytes} [Thu Jun 23 22:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:27:58.992319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8620dce-add2-40f0-aa1c-43b4d6218317 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:58.992564 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8620dce-add2-40f0-aa1c-43b4d6218317 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:58.992848 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8620dce-add2-40f0-aa1c-43b4d6218317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:59.003141 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:59.070549 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c8620dce-add2-40f0-aa1c-43b4d6218317 None None] MFA Rules not processed for user `37ba527b96634fc2a48b6ca9597d8191`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:27:59.071792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c8620dce-add2-40f0-aa1c-43b4d6218317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:59.148302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 644/1315] 10.222.0.43 () {58 vars in 1061 bytes} [Thu Jun 23 22:27:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:27:59.157476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae62362-d967-4b86-b583-7328111c952b None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:27:59.159040 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-6ae62362-d967-4b86-b583-7328111c952b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:59.260882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae62362-d967-4b86-b583-7328111c952b None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:27:59.329544 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:27:59.332653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c8491-eb93-4ed3-b7d7-b643be1871a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:27:59.333540 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c8491-eb93-4ed3-b7d7-b643be1871a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:27:59.333998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c8491-eb93-4ed3-b7d7-b643be1871a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:27:59.334482 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c8491-eb93-4ed3-b7d7-b643be1871a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:27:59.336192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-c01c8491-eb93-4ed3-b7d7-b643be1871a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:27:59.443107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c8491-eb93-4ed3-b7d7-b643be1871a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:27:59.444549 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c8491-eb93-4ed3-b7d7-b643be1871a5 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:27:59.462607 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 672/1316] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:27:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:28:02.019915 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea61d2c-014d-4752-94ab-0f4c66f49c66 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:02.020447 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea61d2c-014d-4752-94ab-0f4c66f49c66 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:02.020575 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea61d2c-014d-4752-94ab-0f4c66f49c66 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:02.021854 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 645/1317] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:28:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:02.026960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84677cbe-4b07-4833-9082-93a905dc2c34 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:02.027395 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84677cbe-4b07-4833-9082-93a905dc2c34 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:02.027540 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84677cbe-4b07-4833-9082-93a905dc2c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:02.091453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-84677cbe-4b07-4833-9082-93a905dc2c34 None None] MFA Rules not processed for user `ae81942c6123431c8ac728c83a3710b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:02.092666 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-84677cbe-4b07-4833-9082-93a905dc2c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:02.143572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:02.188110 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 673/1318] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:02 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:28:02.228222 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea61d2c-014d-4752-94ab-0f4c66f49c66 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:02.229757 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0ea61d2c-014d-4752-94ab-0f4c66f49c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:02.330056 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea61d2c-014d-4752-94ab-0f4c66f49c66 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:02.333281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:02.402043 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5479007-b537-4fbb-984f-2f86ef600e5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:02.402682 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5479007-b537-4fbb-984f-2f86ef600e5c service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:02.402990 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5479007-b537-4fbb-984f-2f86ef600e5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:02.403528 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5479007-b537-4fbb-984f-2f86ef600e5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:02.405155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d5479007-b537-4fbb-984f-2f86ef600e5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:02.511252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5479007-b537-4fbb-984f-2f86ef600e5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:02.512684 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5479007-b537-4fbb-984f-2f86ef600e5c service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:02.531293 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 646/1319] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:28:02 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:28:05.114889 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b9714-b552-4847-8182-b0285571eb91 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:05.115420 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b9714-b552-4847-8182-b0285571eb91 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:05.115538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b9714-b552-4847-8182-b0285571eb91 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:05.116857 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 674/1320] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:05.122017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0eb69-1005-47a6-9e2c-a0f378cb079c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:05.122444 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0eb69-1005-47a6-9e2c-a0f378cb079c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:05.122585 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0eb69-1005-47a6-9e2c-a0f378cb079c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:05.162558 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:05.187701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a9e0eb69-1005-47a6-9e2c-a0f378cb079c None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:05.188903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a9e0eb69-1005-47a6-9e2c-a0f378cb079c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:05.283754 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 647/1321] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:28:05.323858 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b9714-b552-4847-8182-b0285571eb91 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:05.325389 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2f5b9714-b552-4847-8182-b0285571eb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:05.428913 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b9714-b552-4847-8182-b0285571eb91 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:05.499510 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa77c3-6b5b-422d-83ee-d8f6197ab932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:05.500161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa77c3-6b5b-422d-83ee-d8f6197ab932 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:05.500450 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa77c3-6b5b-422d-83ee-d8f6197ab932 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:05.500730 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa77c3-6b5b-422d-83ee-d8f6197ab932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:05.502390 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-56fa77c3-6b5b-422d-83ee-d8f6197ab932 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:05.510312 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:05.612270 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa77c3-6b5b-422d-83ee-d8f6197ab932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:05.614416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa77c3-6b5b-422d-83ee-d8f6197ab932 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:05.631803 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 675/1322] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:28:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:28:08.353230 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9613a8d-1941-45fe-b0f4-20ceb57bda4d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:08.353825 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9613a8d-1941-45fe-b0f4-20ceb57bda4d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:08.353950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9613a8d-1941-45fe-b0f4-20ceb57bda4d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:08.355439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 648/1323] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:08.360627 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2dbd5-dc38-415c-8e85-62b9527f2aa3 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:08.361084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2dbd5-dc38-415c-8e85-62b9527f2aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:08.361187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2dbd5-dc38-415c-8e85-62b9527f2aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:08.427546 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-3ac2dbd5-dc38-415c-8e85-62b9527f2aa3 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:08.428668 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3ac2dbd5-dc38-415c-8e85-62b9527f2aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:08.490309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:08.534289 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 676/1324] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:28:08.571091 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9613a8d-1941-45fe-b0f4-20ceb57bda4d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:08.572783 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f9613a8d-1941-45fe-b0f4-20ceb57bda4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:08.683805 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9613a8d-1941-45fe-b0f4-20ceb57bda4d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:08.695743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:08.756466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa512b02-162a-4700-b7eb-dd2fdcae2bb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:08.757111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa512b02-162a-4700-b7eb-dd2fdcae2bb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:08.757393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa512b02-162a-4700-b7eb-dd2fdcae2bb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:08.757670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa512b02-162a-4700-b7eb-dd2fdcae2bb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:08.759383 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fa512b02-162a-4700-b7eb-dd2fdcae2bb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:08.868036 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa512b02-162a-4700-b7eb-dd2fdcae2bb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:08.869469 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa512b02-162a-4700-b7eb-dd2fdcae2bb7 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:08.888067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 649/1325] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:28:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:28:11.000173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0d0a4-6069-4bb2-84fb-6a76f93d13c8 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:11.000738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0d0a4-6069-4bb2-84fb-6a76f93d13c8 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:11.000738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0d0a4-6069-4bb2-84fb-6a76f93d13c8 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:11.002107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 677/1326] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:11.007500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92208b-8ac5-4649-812f-2fb84df63288 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:11.007929 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92208b-8ac5-4649-812f-2fb84df63288 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:11.008068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd92208b-8ac5-4649-812f-2fb84df63288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:11.056165 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:11.075034 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bd92208b-8ac5-4649-812f-2fb84df63288 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:11.076242 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bd92208b-8ac5-4649-812f-2fb84df63288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:11.174840 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 650/1327] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:28:11.210426 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0d0a4-6069-4bb2-84fb-6a76f93d13c8 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:11.212031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-cef0d0a4-6069-4bb2-84fb-6a76f93d13c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:11.314987 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0d0a4-6069-4bb2-84fb-6a76f93d13c8 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:11.395960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38517843-dae1-4728-bc6a-8494cf9f5e9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:11.396643 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38517843-dae1-4728-bc6a-8494cf9f5e9e service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:11.396932 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38517843-dae1-4728-bc6a-8494cf9f5e9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:11.397214 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38517843-dae1-4728-bc6a-8494cf9f5e9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:11.399040 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-38517843-dae1-4728-bc6a-8494cf9f5e9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:11.419029 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:11.513369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38517843-dae1-4728-bc6a-8494cf9f5e9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:11.514808 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38517843-dae1-4728-bc6a-8494cf9f5e9e service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:11.533150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 678/1328] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:28:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:28:13.694720 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f5df1-7e1b-4611-909a-492bb75c59e2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:13.695255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f5df1-7e1b-4611-909a-492bb75c59e2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:13.695255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f5df1-7e1b-4611-909a-492bb75c59e2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:13.696610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 651/1329] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:13.701911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f31f4f-53c7-4cc4-ae17-f244e6fde488 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:13.702327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f31f4f-53c7-4cc4-ae17-f244e6fde488 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:13.702601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f31f4f-53c7-4cc4-ae17-f244e6fde488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:13.776567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-26f31f4f-53c7-4cc4-ae17-f244e6fde488 None None] MFA Rules not processed for user `ae81942c6123431c8ac728c83a3710b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:13.777806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-26f31f4f-53c7-4cc4-ae17-f244e6fde488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:13.846884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:13.880577 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 679/1330] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:13 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:28:13.919095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f5df1-7e1b-4611-909a-492bb75c59e2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:13.920630 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-715f5df1-7e1b-4611-909a-492bb75c59e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:14.026206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f5df1-7e1b-4611-909a-492bb75c59e2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:14.047183 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:14.103484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94062daf-6312-49f2-bc15-75d1dc93a101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:14.104146 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94062daf-6312-49f2-bc15-75d1dc93a101 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:14.104439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94062daf-6312-49f2-bc15-75d1dc93a101 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:14.104721 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94062daf-6312-49f2-bc15-75d1dc93a101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:14.106380 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-94062daf-6312-49f2-bc15-75d1dc93a101 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:14.238916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94062daf-6312-49f2-bc15-75d1dc93a101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:14.240396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94062daf-6312-49f2-bc15-75d1dc93a101 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:14.259939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 652/1331] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:28:13 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:28:16.531946 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c00d4d-ceb2-46cb-b4e8-666837f6aaf3 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:16.532520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c00d4d-ceb2-46cb-b4e8-666837f6aaf3 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:16.532520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c00d4d-ceb2-46cb-b4e8-666837f6aaf3 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:16.533864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 680/1332] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:16.539082 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c2f22-146e-4b4f-b9e3-e15871630fcf None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:16.539502 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c2f22-146e-4b4f-b9e3-e15871630fcf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:16.539642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c2f22-146e-4b4f-b9e3-e15871630fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:16.601451 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:16.604446 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c66c2f22-146e-4b4f-b9e3-e15871630fcf None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:16.606160 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c66c2f22-146e-4b4f-b9e3-e15871630fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:16.702628 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 653/1333] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:28:16.738646 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c00d4d-ceb2-46cb-b4e8-666837f6aaf3 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:16.740239 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e3c00d4d-ceb2-46cb-b4e8-666837f6aaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:16.841830 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c00d4d-ceb2-46cb-b4e8-666837f6aaf3 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:16.916270 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe66df-4dce-4ca4-9bd5-9ad0681e444d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:16.916923 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe66df-4dce-4ca4-9bd5-9ad0681e444d service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:16.917210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe66df-4dce-4ca4-9bd5-9ad0681e444d service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:16.917491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe66df-4dce-4ca4-9bd5-9ad0681e444d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:16.919161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-eefe66df-4dce-4ca4-9bd5-9ad0681e444d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:16.946022 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:17.029456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe66df-4dce-4ca4-9bd5-9ad0681e444d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:17.030882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe66df-4dce-4ca4-9bd5-9ad0681e444d service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:17.049793 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 681/1334] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:28:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:28:19.127288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eab6a6-c036-4553-8242-47bcdbfefb78 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:19.127694 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eab6a6-c036-4553-8242-47bcdbfefb78 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:19.127884 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eab6a6-c036-4553-8242-47bcdbfefb78 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:19.129993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 654/1335] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:28:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:19.135233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ec03c-76a1-41a1-80a0-4b3ceae65978 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:19.135673 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ec03c-76a1-41a1-80a0-4b3ceae65978 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:19.135770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ec03c-76a1-41a1-80a0-4b3ceae65978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:19.200430 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-279ec03c-76a1-41a1-80a0-4b3ceae65978 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:19.201631 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-279ec03c-76a1-41a1-80a0-4b3ceae65978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:19.273856 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:19.296156 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 682/1336] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:28:19.336530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eab6a6-c036-4553-8242-47bcdbfefb78 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:19.338116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d0eab6a6-c036-4553-8242-47bcdbfefb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:19.440264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eab6a6-c036-4553-8242-47bcdbfefb78 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:19.468597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:19.513452 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed463eda-7b46-4e97-8cca-65af39dd5e8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:19.514098 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed463eda-7b46-4e97-8cca-65af39dd5e8d service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:19.514386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed463eda-7b46-4e97-8cca-65af39dd5e8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:19.514668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed463eda-7b46-4e97-8cca-65af39dd5e8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:19.516333 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ed463eda-7b46-4e97-8cca-65af39dd5e8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:19.622018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed463eda-7b46-4e97-8cca-65af39dd5e8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:19.623387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed463eda-7b46-4e97-8cca-65af39dd5e8d service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:19.641965 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 655/1337] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:28:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:28:21.994639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31b25c-b65d-4a1a-b53a-ddcd7ad95c14 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:21.995240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31b25c-b65d-4a1a-b53a-ddcd7ad95c14 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:21.995240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31b25c-b65d-4a1a-b53a-ddcd7ad95c14 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:21.996665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 683/1338] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:22.001742 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2612a48-6193-4666-a76a-d5bfd8ad9330 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:22.002178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2612a48-6193-4666-a76a-d5bfd8ad9330 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:22.002293 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2612a48-6193-4666-a76a-d5bfd8ad9330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:22.064921 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b2612a48-6193-4666-a76a-d5bfd8ad9330 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:22.066637 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b2612a48-6193-4666-a76a-d5bfd8ad9330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:22.074850 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:22.164882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 656/1339] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:28:22.200366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d31b25c-b65d-4a1a-b53a-ddcd7ad95c14 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:22.201889 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1d31b25c-b65d-4a1a-b53a-ddcd7ad95c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:22.303668 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d31b25c-b65d-4a1a-b53a-ddcd7ad95c14 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:22.375238 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5354f-8dfe-4f21-a0c6-db5b0ca64bd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:22.375889 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5354f-8dfe-4f21-a0c6-db5b0ca64bd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:22.376172 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5354f-8dfe-4f21-a0c6-db5b0ca64bd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:22.376447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5354f-8dfe-4f21-a0c6-db5b0ca64bd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:22.378070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-5fa5354f-8dfe-4f21-a0c6-db5b0ca64bd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:22.414022 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:22.486931 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa5354f-8dfe-4f21-a0c6-db5b0ca64bd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:22.488639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa5354f-8dfe-4f21-a0c6-db5b0ca64bd1 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:22.506406 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 684/1340] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:28:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:28:26.441061 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9ea70-5e3d-45a0-aac1-a957c2bebb0a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:26.441580 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9ea70-5e3d-45a0-aac1-a957c2bebb0a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:26.441580 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9ea70-5e3d-45a0-aac1-a957c2bebb0a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:26.443103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 657/1341] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:26.448541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86af89-5b94-4888-9186-719eaed9c7df None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:26.448987 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86af89-5b94-4888-9186-719eaed9c7df None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:26.449244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86af89-5b94-4888-9186-719eaed9c7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:26.520923 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-8a86af89-5b94-4888-9186-719eaed9c7df None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:26.522261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8a86af89-5b94-4888-9186-719eaed9c7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:26.614691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:26.627996 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 685/1342] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:28:26.667237 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9ea70-5e3d-45a0-aac1-a957c2bebb0a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:26.669445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-99d9ea70-5e3d-45a0-aac1-a957c2bebb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:26.764827 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9ea70-5e3d-45a0-aac1-a957c2bebb0a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:26.768405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14233959-4c13-4bf7-b71b-c3bbc4ee2371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:26.769082 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14233959-4c13-4bf7-b71b-c3bbc4ee2371 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:26.769590 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14233959-4c13-4bf7-b71b-c3bbc4ee2371 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:26.769886 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14233959-4c13-4bf7-b71b-c3bbc4ee2371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:26.779143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14233959-4c13-4bf7-b71b-c3bbc4ee2371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:26.780561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14233959-4c13-4bf7-b71b-c3bbc4ee2371 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:26.789867 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 658/1343] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:28:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:28:29.814579 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca623dd1-1216-4272-beae-58d2e2a83590 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:29.815174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca623dd1-1216-4272-beae-58d2e2a83590 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:29.815174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca623dd1-1216-4272-beae-58d2e2a83590 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:29.817136 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 686/1344] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:28:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:29.822687 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9f920-cc78-4bd9-bc3d-42517bf74982 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:29.823163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9f920-cc78-4bd9-bc3d-42517bf74982 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:29.823303 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9f920-cc78-4bd9-bc3d-42517bf74982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:29.850382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ddb9f920-cc78-4bd9-bc3d-42517bf74982 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:29.851740 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ddb9f920-cc78-4bd9-bc3d-42517bf74982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:29.881893 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:29.932121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 659/1345] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:29 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:28:29.971254 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca623dd1-1216-4272-beae-58d2e2a83590 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:29.973570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-ca623dd1-1216-4272-beae-58d2e2a83590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:29.986280 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca623dd1-1216-4272-beae-58d2e2a83590 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:29.990364 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdf3f8-2b27-481f-b218-7e6473f699ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:29.991036 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdf3f8-2b27-481f-b218-7e6473f699ad service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:29.991322 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdf3f8-2b27-481f-b218-7e6473f699ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:29.991601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdf3f8-2b27-481f-b218-7e6473f699ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:30.000560 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cdf3f8-2b27-481f-b218-7e6473f699ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:30.001976 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cdf3f8-2b27-481f-b218-7e6473f699ad service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:30.012380 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 687/1346] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:28:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:28:34.106131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd6f98-3aaf-43f0-8101-9859fbe83b0d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:34.106638 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd6f98-3aaf-43f0-8101-9859fbe83b0d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:34.106638 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd6f98-3aaf-43f0-8101-9859fbe83b0d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:34.108267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 660/1347] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:34.113567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fa28b-f8e8-41cc-a4b1-d724c1b1d14c None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:34.114050 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fa28b-f8e8-41cc-a4b1-d724c1b1d14c None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:34.114145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fa28b-f8e8-41cc-a4b1-d724c1b1d14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:34.135130 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-607fa28b-f8e8-41cc-a4b1-d724c1b1d14c None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:34.136338 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-607fa28b-f8e8-41cc-a4b1-d724c1b1d14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:34.145011 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 688/1348] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:28:34.183870 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd6f98-3aaf-43f0-8101-9859fbe83b0d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:34.194584 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd6f98-3aaf-43f0-8101-9859fbe83b0d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:34.198342 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017d524-7389-43db-a5de-58803a286d56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:34.199022 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017d524-7389-43db-a5de-58803a286d56 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:34.199308 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017d524-7389-43db-a5de-58803a286d56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:34.199587 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017d524-7389-43db-a5de-58803a286d56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:34.209345 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017d524-7389-43db-a5de-58803a286d56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:34.210784 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017d524-7389-43db-a5de-58803a286d56 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:34.220300 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 661/1349] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:28:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:28:39.305913 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67885ed3-dd2c-437d-b1c3-24298abf2b82 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:39.306872 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67885ed3-dd2c-437d-b1c3-24298abf2b82 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:39.308801 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67885ed3-dd2c-437d-b1c3-24298abf2b82 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:39.310696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 689/1350] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 22:28:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:39.316192 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61181d20-2d88-4cec-83a6-35de87b84ab2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:39.317189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61181d20-2d88-4cec-83a6-35de87b84ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:39.317633 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61181d20-2d88-4cec-83a6-35de87b84ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:39.405903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-61181d20-2d88-4cec-83a6-35de87b84ab2 None None] MFA Rules not processed for user `101864afd59543e699189917391e6ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:39.407139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-61181d20-2d88-4cec-83a6-35de87b84ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:39.498999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 662/1351] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 22:28:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:28:43.756984 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb770b6-4091-40bf-bff1-33917cfcc203 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:43.757574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb770b6-4091-40bf-bff1-33917cfcc203 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:43.757809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb770b6-4091-40bf-bff1-33917cfcc203 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:43.759381 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 690/1352] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 22:28:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:43.765134 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a9ba6-29ce-4611-85bb-7c17337e71c2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:43.766050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a9ba6-29ce-4611-85bb-7c17337e71c2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:43.766525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a9ba6-29ce-4611-85bb-7c17337e71c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:43.786179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-103a9ba6-29ce-4611-85bb-7c17337e71c2 None None] MFA Rules not processed for user `101864afd59543e699189917391e6ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:43.787476 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-103a9ba6-29ce-4611-85bb-7c17337e71c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:43.796455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 663/1353] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 22:28:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:28:44.127462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=13 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:45.325353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6b91f-9deb-4476-ae0c-79061e20efab None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:45.325932 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6b91f-9deb-4476-ae0c-79061e20efab None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:45.325932 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6b91f-9deb-4476-ae0c-79061e20efab None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:45.327217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 691/1354] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 22:28:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:45.332777 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e3785-da94-487a-8d47-7866b876a560 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:45.333355 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e3785-da94-487a-8d47-7866b876a560 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:45.333355 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e3785-da94-487a-8d47-7866b876a560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:45.352496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2e6e3785-da94-487a-8d47-7866b876a560 None None] MFA Rules not processed for user `101864afd59543e699189917391e6ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:45.353708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2e6e3785-da94-487a-8d47-7866b876a560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:45.363966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 664/1355] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 22:28:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:28:49.173407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e94ff6-bf9f-46f5-baa4-44dc302b6d06 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:49.174250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e94ff6-bf9f-46f5-baa4-44dc302b6d06 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:49.175291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e94ff6-bf9f-46f5-baa4-44dc302b6d06 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:49.176971 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 692/1356] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:28:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:49.182657 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca9c35-aa65-45bb-8283-416546a87d1f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:49.183179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca9c35-aa65-45bb-8283-416546a87d1f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:49.183278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca9c35-aa65-45bb-8283-416546a87d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:49.217811 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c1ca9c35-aa65-45bb-8283-416546a87d1f None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:49.217811 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c1ca9c35-aa65-45bb-8283-416546a87d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:49.222974 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 665/1357] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:28:49.278570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546e896-aa31-4e3d-ad11-346f355de6b4 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:49.279391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546e896-aa31-4e3d-ad11-346f355de6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:49.279842 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546e896-aa31-4e3d-ad11-346f355de6b4 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:49.281960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 693/1358] 10.222.0.43 () {54 vars in 961 bytes} [Thu Jun 23 22:28:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:49.293375 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8648a-7a3e-4bad-a73f-d3cf08142f2c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:49.294099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8648a-7a3e-4bad-a73f-d3cf08142f2c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:49.294773 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8648a-7a3e-4bad-a73f-d3cf08142f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:49.341621 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:49.352564 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fef8648a-7a3e-4bad-a73f-d3cf08142f2c None None] MFA Rules not processed for user `35d686923ef541819de3c77847d38511`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:49.353890 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fef8648a-7a3e-4bad-a73f-d3cf08142f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:49.455736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 666/1359] 10.222.0.43 () {58 vars in 1058 bytes} [Thu Jun 23 22:28:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:28:49.465364 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546e896-aa31-4e3d-ad11-346f355de6b4 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:49.480879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546e896-aa31-4e3d-ad11-346f355de6b4 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:49.486720 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72396c58-7de8-4e58-baec-76201a05e8f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:49.488234 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72396c58-7de8-4e58-baec-76201a05e8f4 service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:49.488697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72396c58-7de8-4e58-baec-76201a05e8f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:49.489139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72396c58-7de8-4e58-baec-76201a05e8f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:49.498233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72396c58-7de8-4e58-baec-76201a05e8f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:49.500010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72396c58-7de8-4e58-baec-76201a05e8f4 service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:49.510950 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 694/1360] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:28:51.783581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f399445-6045-45d8-a79c-54cd94cc3de4 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:51.784179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f399445-6045-45d8-a79c-54cd94cc3de4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:51.784179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f399445-6045-45d8-a79c-54cd94cc3de4 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:51.785696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 667/1361] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:51.790765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14bc6c-4cca-40a1-852d-0745a139f706 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:51.791220 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14bc6c-4cca-40a1-852d-0745a139f706 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:51.791363 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14bc6c-4cca-40a1-852d-0745a139f706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:51.809673 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-6e14bc6c-4cca-40a1-852d-0745a139f706 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:51.810862 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-6e14bc6c-4cca-40a1-852d-0745a139f706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:51.819621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 695/1362] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:28:51.865701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f399445-6045-45d8-a79c-54cd94cc3de4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:51.881135 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f399445-6045-45d8-a79c-54cd94cc3de4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:51.886750 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f2f2e-ce44-4a1a-869f-6c6daf5c4f0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:51.887429 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f2f2e-ce44-4a1a-869f-6c6daf5c4f0c service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:51.887722 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f2f2e-ce44-4a1a-869f-6c6daf5c4f0c service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:51.888007 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f2f2e-ce44-4a1a-869f-6c6daf5c4f0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:51.898269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f2f2e-ce44-4a1a-869f-6c6daf5c4f0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:51.899890 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f2f2e-ce44-4a1a-869f-6c6daf5c4f0c service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:51.910676 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 668/1363] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:28:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:28:54.344077 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934434b7-3bb9-4140-b86f-60d1414d7546 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:54.345017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934434b7-3bb9-4140-b86f-60d1414d7546 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:54.345514 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934434b7-3bb9-4140-b86f-60d1414d7546 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:54.348154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 696/1364] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:28:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:54.353903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241cad4-433d-4bd9-a2b3-3864461c7a32 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:54.354815 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241cad4-433d-4bd9-a2b3-3864461c7a32 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:54.355344 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241cad4-433d-4bd9-a2b3-3864461c7a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:54.375084 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3241cad4-433d-4bd9-a2b3-3864461c7a32 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:54.376306 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3241cad4-433d-4bd9-a2b3-3864461c7a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:54.384560 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 669/1365] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:28:54.430624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505c300-fb26-4b26-8e25-ab6d9dec80d6 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:54.430930 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505c300-fb26-4b26-8e25-ab6d9dec80d6 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:54.431232 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505c300-fb26-4b26-8e25-ab6d9dec80d6 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:54.432584 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 697/1366] 10.222.0.43 () {54 vars in 961 bytes} [Thu Jun 23 22:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:54.437781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16a06f-d444-48c5-b219-54bcc60c5faa None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:54.438072 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16a06f-d444-48c5-b219-54bcc60c5faa None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:54.438365 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16a06f-d444-48c5-b219-54bcc60c5faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:54.457313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0e16a06f-d444-48c5-b219-54bcc60c5faa None None] MFA Rules not processed for user `35d686923ef541819de3c77847d38511`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:54.458446 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0e16a06f-d444-48c5-b219-54bcc60c5faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:54.468154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 670/1367] 10.222.0.43 () {58 vars in 1058 bytes} [Thu Jun 23 22:28:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:28:54.476950 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505c300-fb26-4b26-8e25-ab6d9dec80d6 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:54.491634 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505c300-fb26-4b26-8e25-ab6d9dec80d6 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:54.497076 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b9fd2-60bc-452d-9194-fbbf8b46c700 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:54.497717 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b9fd2-60bc-452d-9194-fbbf8b46c700 service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:54.498002 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b9fd2-60bc-452d-9194-fbbf8b46c700 service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:54.498282 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b9fd2-60bc-452d-9194-fbbf8b46c700 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:54.506645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b9fd2-60bc-452d-9194-fbbf8b46c700 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:54.508260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b9fd2-60bc-452d-9194-fbbf8b46c700 service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:54.517578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 698/1368] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:28:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:28:56.653107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92e168-a4d2-4515-b30a-236cb8f87fec None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:56.654433 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92e168-a4d2-4515-b30a-236cb8f87fec None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:56.654994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92e168-a4d2-4515-b30a-236cb8f87fec None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:56.656698 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 671/1369] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:28:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:56.662957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f32b1-eef6-4193-8a60-63d8a4b0fa64 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:56.663532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f32b1-eef6-4193-8a60-63d8a4b0fa64 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:56.663532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f32b1-eef6-4193-8a60-63d8a4b0fa64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:56.681781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-766f32b1-eef6-4193-8a60-63d8a4b0fa64 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:56.683007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-766f32b1-eef6-4193-8a60-63d8a4b0fa64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:56.691695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 699/1370] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:28:56.735615 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92e168-a4d2-4515-b30a-236cb8f87fec None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:56.750751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92e168-a4d2-4515-b30a-236cb8f87fec None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:56.757507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88603fdc-9f88-4176-a869-ed6cb3d9bfb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:56.758168 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88603fdc-9f88-4176-a869-ed6cb3d9bfb9 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:56.758598 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88603fdc-9f88-4176-a869-ed6cb3d9bfb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:56.759068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88603fdc-9f88-4176-a869-ed6cb3d9bfb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:56.768816 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88603fdc-9f88-4176-a869-ed6cb3d9bfb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:56.770650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88603fdc-9f88-4176-a869-ed6cb3d9bfb9 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:56.781181 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 672/1371] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:28:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:28:57.372688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070506a-ab8d-4e1e-9a06-d6e47494198a None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:57.373206 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070506a-ab8d-4e1e-9a06-d6e47494198a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:57.373638 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070506a-ab8d-4e1e-9a06-d6e47494198a None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:57.375431 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 700/1372] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:28:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:57.380565 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026aa45-3e3e-4a48-a705-5ce94414a238 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:57.380925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026aa45-3e3e-4a48-a705-5ce94414a238 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:57.381276 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026aa45-3e3e-4a48-a705-5ce94414a238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:57.400140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0026aa45-3e3e-4a48-a705-5ce94414a238 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:57.401550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0026aa45-3e3e-4a48-a705-5ce94414a238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:57.414390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 673/1373] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:28:57.761249 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a65c3-ebb7-40aa-922d-762a528f43a9 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:57.761249 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a65c3-ebb7-40aa-922d-762a528f43a9 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:57.761249 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a65c3-ebb7-40aa-922d-762a528f43a9 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:57.762041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 701/1374] 10.222.0.43 () {54 vars in 963 bytes} [Thu Jun 23 22:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:57.766449 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9822194-0f70-4e1a-9ffc-a0151f09eec5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:57.766901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9822194-0f70-4e1a-9ffc-a0151f09eec5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:57.767240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9822194-0f70-4e1a-9ffc-a0151f09eec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:57.826346 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d9822194-0f70-4e1a-9ffc-a0151f09eec5 None None] MFA Rules not processed for user `95bc067d93fa4207bec1c85600728085`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:57.827779 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d9822194-0f70-4e1a-9ffc-a0151f09eec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:57.901758 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 674/1375] 10.222.0.43 () {58 vars in 1060 bytes} [Thu Jun 23 22:28:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:28:57.910438 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a65c3-ebb7-40aa-922d-762a528f43a9 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:57.921153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a65c3-ebb7-40aa-922d-762a528f43a9 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:57.925325 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010f9e4-e7e8-452a-b2fa-416ef13dd0f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:57.926199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010f9e4-e7e8-452a-b2fa-416ef13dd0f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:57.926663 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010f9e4-e7e8-452a-b2fa-416ef13dd0f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:57.926970 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010f9e4-e7e8-452a-b2fa-416ef13dd0f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:57.943298 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010f9e4-e7e8-452a-b2fa-416ef13dd0f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:57.944780 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010f9e4-e7e8-452a-b2fa-416ef13dd0f0 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:57.960049 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 702/1376] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:28:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:28:58.917121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b3c56-f553-41e1-94bf-7a5a04c5584d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:58.917699 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b3c56-f553-41e1-94bf-7a5a04c5584d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:58.918301 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b3c56-f553-41e1-94bf-7a5a04c5584d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:58.920960 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 675/1377] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:28:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:28:58.926981 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62b46a-2c61-40ae-a182-d9fccb71e700 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:58.927474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62b46a-2c61-40ae-a182-d9fccb71e700 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:58.927580 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62b46a-2c61-40ae-a182-d9fccb71e700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:58.946171 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-fb62b46a-2c61-40ae-a182-d9fccb71e700 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:28:58.947434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-fb62b46a-2c61-40ae-a182-d9fccb71e700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:28:58.959369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 703/1378] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:28:59.009318 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b3c56-f553-41e1-94bf-7a5a04c5584d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:28:59.024382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b3c56-f553-41e1-94bf-7a5a04c5584d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:28:59.029999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe9c7b-e874-4923-8eb9-eb1932ed6dd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:28:59.030636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe9c7b-e874-4923-8eb9-eb1932ed6dd2 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:28:59.030947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe9c7b-e874-4923-8eb9-eb1932ed6dd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:28:59.031227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe9c7b-e874-4923-8eb9-eb1932ed6dd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:28:59.041994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe9c7b-e874-4923-8eb9-eb1932ed6dd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:28:59.043607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe9c7b-e874-4923-8eb9-eb1932ed6dd2 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:28:59.047541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:28:59.054439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 676/1379] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:28:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:01.308632 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb32d7-4b0d-4752-94fa-784ea9e0bc6f None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:01.309140 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb32d7-4b0d-4752-94fa-784ea9e0bc6f None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:01.309140 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb32d7-4b0d-4752-94fa-784ea9e0bc6f None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:01.310512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 704/1380] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:01.316665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52674d-737d-467b-a7e6-e3e03c8e25df None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:01.317122 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52674d-737d-467b-a7e6-e3e03c8e25df None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:01.317214 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52674d-737d-467b-a7e6-e3e03c8e25df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:01.335306 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-df52674d-737d-467b-a7e6-e3e03c8e25df None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:01.336506 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-df52674d-737d-467b-a7e6-e3e03c8e25df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:01.344303 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 677/1381] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:01.389318 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb32d7-4b0d-4752-94fa-784ea9e0bc6f None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:01.403745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb32d7-4b0d-4752-94fa-784ea9e0bc6f None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:01.409909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b58f7-fcbc-44f9-94b3-2e691c1adbc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:01.410545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b58f7-fcbc-44f9-94b3-2e691c1adbc7 service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:01.410844 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b58f7-fcbc-44f9-94b3-2e691c1adbc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:01.411155 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b58f7-fcbc-44f9-94b3-2e691c1adbc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:01.419516 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b58f7-fcbc-44f9-94b3-2e691c1adbc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:01.421102 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b58f7-fcbc-44f9-94b3-2e691c1adbc7 service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:01.431442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 705/1382] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:03.605258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80935d-d149-4f0a-8234-e6062eb69586 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:03.605816 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80935d-d149-4f0a-8234-e6062eb69586 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:03.605816 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80935d-d149-4f0a-8234-e6062eb69586 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:03.607483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 678/1383] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:03.613003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b1d15-1ae4-4d5b-894c-e914a45b78ec None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:03.613437 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b1d15-1ae4-4d5b-894c-e914a45b78ec None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:03.613623 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b1d15-1ae4-4d5b-894c-e914a45b78ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:03.634395 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-062b1d15-1ae4-4d5b-894c-e914a45b78ec None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:03.635640 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-062b1d15-1ae4-4d5b-894c-e914a45b78ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:03.646746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 706/1384] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:03.693123 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80935d-d149-4f0a-8234-e6062eb69586 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:03.708257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80935d-d149-4f0a-8234-e6062eb69586 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:03.714140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac42184-3b0f-46b7-a6fd-03032cb81af1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:03.715027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac42184-3b0f-46b7-a6fd-03032cb81af1 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:03.715537 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac42184-3b0f-46b7-a6fd-03032cb81af1 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:03.716046 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac42184-3b0f-46b7-a6fd-03032cb81af1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:03.726462 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac42184-3b0f-46b7-a6fd-03032cb81af1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:03.728073 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac42184-3b0f-46b7-a6fd-03032cb81af1 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:03.740405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 679/1385] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:05.929430 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916726a-0c1f-425b-bb39-7649165986f3 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:05.929866 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916726a-0c1f-425b-bb39-7649165986f3 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:05.929866 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916726a-0c1f-425b-bb39-7649165986f3 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:05.931777 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 707/1386] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:05.938537 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279ecdf-eebc-443e-b95e-e3018264cd85 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:05.938957 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279ecdf-eebc-443e-b95e-e3018264cd85 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:05.939984 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279ecdf-eebc-443e-b95e-e3018264cd85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:05.964231 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f279ecdf-eebc-443e-b95e-e3018264cd85 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:05.965593 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f279ecdf-eebc-443e-b95e-e3018264cd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:05.979165 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 680/1387] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:06.022527 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916726a-0c1f-425b-bb39-7649165986f3 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:06.040984 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916726a-0c1f-425b-bb39-7649165986f3 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:06.046616 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493f795-2ec4-493d-9f83-44d2969dd5aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:06.047367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493f795-2ec4-493d-9f83-44d2969dd5aa service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:06.047718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493f795-2ec4-493d-9f83-44d2969dd5aa service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:06.048004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493f795-2ec4-493d-9f83-44d2969dd5aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:06.057152 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493f795-2ec4-493d-9f83-44d2969dd5aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:06.058751 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493f795-2ec4-493d-9f83-44d2969dd5aa service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:06.068915 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 708/1388] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:08.348339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ef3c2-8e98-4c2d-a870-38ec63e4eb82 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:08.349084 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ef3c2-8e98-4c2d-a870-38ec63e4eb82 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:08.349084 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ef3c2-8e98-4c2d-a870-38ec63e4eb82 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:08.350251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 681/1389] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:29:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:08.356145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563ead6-0b9a-4e51-b8eb-96ffbd841463 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:08.356633 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563ead6-0b9a-4e51-b8eb-96ffbd841463 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:08.356855 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563ead6-0b9a-4e51-b8eb-96ffbd841463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:08.379867 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-5563ead6-0b9a-4e51-b8eb-96ffbd841463 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:08.381637 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-5563ead6-0b9a-4e51-b8eb-96ffbd841463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:08.391917 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 709/1390] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:08.438987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ef3c2-8e98-4c2d-a870-38ec63e4eb82 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:08.458238 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ef3c2-8e98-4c2d-a870-38ec63e4eb82 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:08.464290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17109949-46fe-4dca-b2fb-600929d93b44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:08.464794 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17109949-46fe-4dca-b2fb-600929d93b44 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:08.465085 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17109949-46fe-4dca-b2fb-600929d93b44 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:08.465366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17109949-46fe-4dca-b2fb-600929d93b44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:08.474340 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17109949-46fe-4dca-b2fb-600929d93b44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:08.475968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17109949-46fe-4dca-b2fb-600929d93b44 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:08.485757 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 682/1391] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:11.007844 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0a689-127c-4643-9be9-89d01904837f None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:11.008519 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0a689-127c-4643-9be9-89d01904837f None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:11.008519 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0a689-127c-4643-9be9-89d01904837f None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:11.010264 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 710/1392] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:29:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:11.015570 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db846c7-3787-4fde-9515-7374a479fcb4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:11.016005 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db846c7-3787-4fde-9515-7374a479fcb4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:11.016139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db846c7-3787-4fde-9515-7374a479fcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:11.034668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9db846c7-3787-4fde-9515-7374a479fcb4 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:11.035947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9db846c7-3787-4fde-9515-7374a479fcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:11.044172 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 683/1393] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:29:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:11.090209 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0a689-127c-4643-9be9-89d01904837f None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:11.109399 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0a689-127c-4643-9be9-89d01904837f None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:11.117522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a91568-dca7-4692-bfd4-44264c6c5875 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:11.118496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a91568-dca7-4692-bfd4-44264c6c5875 service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:11.118879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a91568-dca7-4692-bfd4-44264c6c5875 service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:11.119323 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a91568-dca7-4692-bfd4-44264c6c5875 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:11.132435 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a91568-dca7-4692-bfd4-44264c6c5875 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:11.134770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a91568-dca7-4692-bfd4-44264c6c5875 service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:11.148292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 711/1394] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:13.261019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd61cd-774e-43cb-a247-d059c7640719 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:13.261594 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd61cd-774e-43cb-a247-d059c7640719 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:13.261594 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd61cd-774e-43cb-a247-d059c7640719 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:13.262981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 684/1395] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:29:13.269368 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4a289-3815-4974-aabb-b7423711938d None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:13.269796 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4a289-3815-4974-aabb-b7423711938d None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:13.269979 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4a289-3815-4974-aabb-b7423711938d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:13.289083 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-d8f4a289-3815-4974-aabb-b7423711938d None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:13.290276 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d8f4a289-3815-4974-aabb-b7423711938d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:13.298050 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 712/1396] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:29:13.343567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd61cd-774e-43cb-a247-d059c7640719 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:13.358880 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd61cd-774e-43cb-a247-d059c7640719 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:13.364508 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f677367-bce8-438c-8edb-0ca8c0270173 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:13.365162 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f677367-bce8-438c-8edb-0ca8c0270173 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:13.365453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f677367-bce8-438c-8edb-0ca8c0270173 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:13.365733 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f677367-bce8-438c-8edb-0ca8c0270173 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:13.374213 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f677367-bce8-438c-8edb-0ca8c0270173 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:13.375806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f677367-bce8-438c-8edb-0ca8c0270173 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:13.385231 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 685/1397] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:13.652773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2705082-bb2b-4eb2-965c-c04ce718c527 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:13.652982 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2705082-bb2b-4eb2-965c-c04ce718c527 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:13.653245 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2705082-bb2b-4eb2-965c-c04ce718c527 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:13.654869 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 713/1398] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:13.659990 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5776f00-435b-4cc0-b65f-49c6e2415ad2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:13.660286 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5776f00-435b-4cc0-b65f-49c6e2415ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:13.660569 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5776f00-435b-4cc0-b65f-49c6e2415ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:13.679005 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a5776f00-435b-4cc0-b65f-49c6e2415ad2 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:13.680211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a5776f00-435b-4cc0-b65f-49c6e2415ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:13.690672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 686/1399] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:13.746042 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b0eca-c2b7-408a-a862-a46c9303e373 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:13.746414 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b0eca-c2b7-408a-a862-a46c9303e373 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:13.746760 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b0eca-c2b7-408a-a862-a46c9303e373 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:13.748226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 714/1400] 10.222.0.43 () {54 vars in 962 bytes} [Thu Jun 23 22:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:13.753959 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558aaea5-6514-4ecb-a8f5-78e6e09cdae8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:13.754330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558aaea5-6514-4ecb-a8f5-78e6e09cdae8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:13.754673 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558aaea5-6514-4ecb-a8f5-78e6e09cdae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:13.803074 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-558aaea5-6514-4ecb-a8f5-78e6e09cdae8 None None] MFA Rules not processed for user `36de21e0b3de48dabe0f27a9db8a37aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:13.804266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-558aaea5-6514-4ecb-a8f5-78e6e09cdae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:13.864925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 687/1401] 10.222.0.43 () {58 vars in 1059 bytes} [Thu Jun 23 22:29:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:13.873413 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b0eca-c2b7-408a-a862-a46c9303e373 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:13.884784 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b0eca-c2b7-408a-a862-a46c9303e373 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:13.888273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed2d67-f9e2-463f-a942-b89adcc6a863 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36de21e0b3de48dabe0f27a9db8a37aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:13.888912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed2d67-f9e2-463f-a942-b89adcc6a863 service glance] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:13.889194 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed2d67-f9e2-463f-a942-b89adcc6a863 service glance] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:13.889469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed2d67-f9e2-463f-a942-b89adcc6a863 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:13.900176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed2d67-f9e2-463f-a942-b89adcc6a863 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:13.901618 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed2d67-f9e2-463f-a942-b89adcc6a863 service glance] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:13.911752 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 715/1402] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:14.638322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da87e4-ef23-4a22-9a7c-0631412ff126 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:14.638772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da87e4-ef23-4a22-9a7c-0631412ff126 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:14.638877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da87e4-ef23-4a22-9a7c-0631412ff126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:14.687501 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-00da87e4-ef23-4a22-9a7c-0631412ff126 None None] MFA Rules not processed for user `df1aeb2be41c49819b811ecfa47d4155`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:14.688708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-00da87e4-ef23-4a22-9a7c-0631412ff126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:14.729567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:14.754400 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 688/1403] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 22:29:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:14.800124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d39b41-d21c-4e6f-9d8a-d5699d079061 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:14.800597 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d39b41-d21c-4e6f-9d8a-d5699d079061 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:14.800696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d39b41-d21c-4e6f-9d8a-d5699d079061 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:14.802115 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 716/1404] 10.222.0.43 () {54 vars in 942 bytes} [Thu Jun 23 22:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:14.808442 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923212c-2d19-4885-a5f2-a53ffda33350 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:14.808739 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923212c-2d19-4885-a5f2-a53ffda33350 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:14.809026 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923212c-2d19-4885-a5f2-a53ffda33350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:14.858210 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a923212c-2d19-4885-a5f2-a53ffda33350 None None] MFA Rules not processed for user `8ce35c43e8294f4cbc880f2219a6eb76`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:14.859386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a923212c-2d19-4885-a5f2-a53ffda33350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:14.934126 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 689/1405] 10.222.0.43 () {58 vars in 1039 bytes} [Thu Jun 23 22:29:14 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:14.943733 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d39b41-d21c-4e6f-9d8a-d5699d079061 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:14.951299 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=61 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:14.959247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d39b41-d21c-4e6f-9d8a-d5699d079061 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:14.964301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca21da-074b-4d22-bde7-5117e8dc8cf5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ce35c43e8294f4cbc880f2219a6eb76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:14.964961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca21da-074b-4d22-bde7-5117e8dc8cf5 service swift] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:14.965248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca21da-074b-4d22-bde7-5117e8dc8cf5 service swift] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:14.965532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca21da-074b-4d22-bde7-5117e8dc8cf5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:14.976391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca21da-074b-4d22-bde7-5117e8dc8cf5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:14.977999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca21da-074b-4d22-bde7-5117e8dc8cf5 service swift] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:14.986494 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 717/1406] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 22:29:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 22:29:15.496132 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d95ee8-de7d-4493-9980-dfaf96c4720c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:15.496715 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d95ee8-de7d-4493-9980-dfaf96c4720c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:15.497113 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d95ee8-de7d-4493-9980-dfaf96c4720c None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:15.498508 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 690/1407] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:29:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:15.504674 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dcbeb-ed1f-40db-b857-9d5a617833f2 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:15.505158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dcbeb-ed1f-40db-b857-9d5a617833f2 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:15.505615 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dcbeb-ed1f-40db-b857-9d5a617833f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:15.528079 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-717dcbeb-ed1f-40db-b857-9d5a617833f2 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:15.529570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-717dcbeb-ed1f-40db-b857-9d5a617833f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:15.538308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 718/1408] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:15.586887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d95ee8-de7d-4493-9980-dfaf96c4720c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:15.605130 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d95ee8-de7d-4493-9980-dfaf96c4720c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:15.610888 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9be38-1d19-4f17-aec7-8901c1893691 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:15.612215 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9be38-1d19-4f17-aec7-8901c1893691 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:15.612652 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9be38-1d19-4f17-aec7-8901c1893691 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:15.612933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9be38-1d19-4f17-aec7-8901c1893691 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:15.621738 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9be38-1d19-4f17-aec7-8901c1893691 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:15.623389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9be38-1d19-4f17-aec7-8901c1893691 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:15.634941 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 691/1409] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:25.519452 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=8 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:25.630118 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=59 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:29:37.305883 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24920e68-a78e-4d00-bde5-945c5643559c None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:37.306447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24920e68-a78e-4d00-bde5-945c5643559c None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:37.306447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24920e68-a78e-4d00-bde5-945c5643559c None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:37.307861 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 719/1410] 10.222.0.43 () {54 vars in 918 bytes} [Thu Jun 23 22:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:37.313121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914cf13-35c9-4bc5-96a9-dd65f6c930ef None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:37.313545 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914cf13-35c9-4bc5-96a9-dd65f6c930ef None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:37.313679 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914cf13-35c9-4bc5-96a9-dd65f6c930ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:37.333342 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9914cf13-35c9-4bc5-96a9-dd65f6c930ef None None] MFA Rules not processed for user `101864afd59543e699189917391e6ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:37.335355 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9914cf13-35c9-4bc5-96a9-dd65f6c930ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:37.347088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 692/1411] 10.222.0.43 () {58 vars in 1015 bytes} [Thu Jun 23 22:29:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:37.360188 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2c7c7-c859-45f0-a8a3-72b47a828ec2 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:37.360188 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2c7c7-c859-45f0-a8a3-72b47a828ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:37.360188 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2c7c7-c859-45f0-a8a3-72b47a828ec2 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:37.361467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 720/1412] 10.222.0.43 () {54 vars in 918 bytes} [Thu Jun 23 22:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:29:37.366996 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b5258-520b-4364-b7a6-6ec63831defb None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:37.367366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b5258-520b-4364-b7a6-6ec63831defb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:37.367661 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b5258-520b-4364-b7a6-6ec63831defb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:37.387985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-760b5258-520b-4364-b7a6-6ec63831defb None None] MFA Rules not processed for user `35d686923ef541819de3c77847d38511`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:37.389206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-760b5258-520b-4364-b7a6-6ec63831defb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:37.399373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 693/1413] 10.222.0.43 () {58 vars in 1015 bytes} [Thu Jun 23 22:29:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:39.882393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5be21-a3fb-4e3c-87a4-0dc0eae70453 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:39.882971 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5be21-a3fb-4e3c-87a4-0dc0eae70453 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:39.883090 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5be21-a3fb-4e3c-87a4-0dc0eae70453 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:39.884390 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 721/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:39.889502 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55320b-7a82-4b53-bc74-ad4a380a75d3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:39.889934 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55320b-7a82-4b53-bc74-ad4a380a75d3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:39.890073 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55320b-7a82-4b53-bc74-ad4a380a75d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:39.908901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-df55320b-7a82-4b53-bc74-ad4a380a75d3 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:39.910121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-df55320b-7a82-4b53-bc74-ad4a380a75d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:39.917930 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 694/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:39.961552 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5be21-a3fb-4e3c-87a4-0dc0eae70453 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:39.976374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5be21-a3fb-4e3c-87a4-0dc0eae70453 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:39.981641 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba78bfe-819b-4114-b83e-de5e21913c54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:39.982278 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba78bfe-819b-4114-b83e-de5e21913c54 service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:39.982561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba78bfe-819b-4114-b83e-de5e21913c54 service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:39.982840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba78bfe-819b-4114-b83e-de5e21913c54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:39.991009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba78bfe-819b-4114-b83e-de5e21913c54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:39.992591 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba78bfe-819b-4114-b83e-de5e21913c54 service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:40.001698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 722/1416] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:40.900004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f5b4f-b80c-410f-b584-3a46f97a878f None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:40.900564 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f5b4f-b80c-410f-b584-3a46f97a878f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:40.900564 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f5b4f-b80c-410f-b584-3a46f97a878f None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:40.901925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 695/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:40.907306 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b252b-ad3a-43ac-a88a-b8c65b6c623f None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:40.907777 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b252b-ad3a-43ac-a88a-b8c65b6c623f None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:40.907876 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b252b-ad3a-43ac-a88a-b8c65b6c623f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:40.926579 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-228b252b-ad3a-43ac-a88a-b8c65b6c623f None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:40.927815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-228b252b-ad3a-43ac-a88a-b8c65b6c623f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:40.935655 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 723/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:41.087473 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f5b4f-b80c-410f-b584-3a46f97a878f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:41.101705 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f5b4f-b80c-410f-b584-3a46f97a878f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:41.107029 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de15a03c-ec22-49ff-a05f-cf71891dd38f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:41.107668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15a03c-ec22-49ff-a05f-cf71891dd38f service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:41.107955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15a03c-ec22-49ff-a05f-cf71891dd38f service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:41.108242 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de15a03c-ec22-49ff-a05f-cf71891dd38f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:41.116870 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15a03c-ec22-49ff-a05f-cf71891dd38f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:41.118435 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de15a03c-ec22-49ff-a05f-cf71891dd38f service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:41.128187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 696/1419] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:42.009875 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761325b-25ea-4ef1-aa9c-638beed4a836 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:42.010449 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761325b-25ea-4ef1-aa9c-638beed4a836 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:42.010449 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761325b-25ea-4ef1-aa9c-638beed4a836 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:42.011939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 724/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:42.016900 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfcc2bc-07b6-4ce6-8912-feb8c9e82151 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:42.017281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfcc2bc-07b6-4ce6-8912-feb8c9e82151 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:42.017470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfcc2bc-07b6-4ce6-8912-feb8c9e82151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:42.036746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-adfcc2bc-07b6-4ce6-8912-feb8c9e82151 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:42.037941 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-adfcc2bc-07b6-4ce6-8912-feb8c9e82151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:42.045700 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 697/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:42.199049 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761325b-25ea-4ef1-aa9c-638beed4a836 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:42.213484 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761325b-25ea-4ef1-aa9c-638beed4a836 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:42.218759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a96f5-e1c9-45bd-8487-7239c2c92d36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:42.219424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a96f5-e1c9-45bd-8487-7239c2c92d36 service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:42.219714 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a96f5-e1c9-45bd-8487-7239c2c92d36 service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:42.220097 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a96f5-e1c9-45bd-8487-7239c2c92d36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:42.228620 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a96f5-e1c9-45bd-8487-7239c2c92d36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:42.230196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a96f5-e1c9-45bd-8487-7239c2c92d36 service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:42.240136 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 725/1422] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:43.107806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af327e-0b95-469a-bc85-ceb411d13149 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:43.108262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af327e-0b95-469a-bc85-ceb411d13149 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:43.108368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af327e-0b95-469a-bc85-ceb411d13149 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:43.109746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 698/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:43.115465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7289e-a8ca-46cb-a8f8-1e7e4bea0832 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:43.115902 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7289e-a8ca-46cb-a8f8-1e7e4bea0832 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:43.116027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7289e-a8ca-46cb-a8f8-1e7e4bea0832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:43.135777 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-e6d7289e-a8ca-46cb-a8f8-1e7e4bea0832 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:43.136985 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e6d7289e-a8ca-46cb-a8f8-1e7e4bea0832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:43.144788 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 726/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:43.298842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af327e-0b95-469a-bc85-ceb411d13149 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:43.313019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af327e-0b95-469a-bc85-ceb411d13149 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:43.318434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020e463a-8d7e-450a-84df-64e4f13558dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:43.319115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e463a-8d7e-450a-84df-64e4f13558dd service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:43.319427 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e463a-8d7e-450a-84df-64e4f13558dd service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:43.319696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e463a-8d7e-450a-84df-64e4f13558dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:43.327755 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020e463a-8d7e-450a-84df-64e4f13558dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:43.329334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020e463a-8d7e-450a-84df-64e4f13558dd service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:43.339480 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 699/1425] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:44.201651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261cac7-d60c-4fb1-884d-d4d05a7d5408 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:44.202187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261cac7-d60c-4fb1-884d-d4d05a7d5408 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:44.202283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261cac7-d60c-4fb1-884d-d4d05a7d5408 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:44.203622 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 727/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:44.208953 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40368b-fc51-4884-b9e5-68eaacaa74a1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:44.209363 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40368b-fc51-4884-b9e5-68eaacaa74a1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:44.209519 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40368b-fc51-4884-b9e5-68eaacaa74a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:44.227222 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-da40368b-fc51-4884-b9e5-68eaacaa74a1 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:44.228419 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-da40368b-fc51-4884-b9e5-68eaacaa74a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:44.236109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 700/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:44.387847 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261cac7-d60c-4fb1-884d-d4d05a7d5408 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:44.402027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261cac7-d60c-4fb1-884d-d4d05a7d5408 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:44.407604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93836a8-0f66-46eb-a0b0-a7651abddd3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:44.408251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93836a8-0f66-46eb-a0b0-a7651abddd3c service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:44.408541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93836a8-0f66-46eb-a0b0-a7651abddd3c service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:44.408822 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93836a8-0f66-46eb-a0b0-a7651abddd3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:44.417748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93836a8-0f66-46eb-a0b0-a7651abddd3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:44.419387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93836a8-0f66-46eb-a0b0-a7651abddd3c service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:44.429416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 728/1428] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:45.232404 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337e8c6-a020-4812-9215-c0a0fbfcc3f9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:45.232958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337e8c6-a020-4812-9215-c0a0fbfcc3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:45.232958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337e8c6-a020-4812-9215-c0a0fbfcc3f9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:45.234350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 701/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:45.239662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdceca-a85c-4eda-8e21-7e64c0cc08b2 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:45.240099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdceca-a85c-4eda-8e21-7e64c0cc08b2 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:45.240214 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdceca-a85c-4eda-8e21-7e64c0cc08b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:45.259790 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-3ecdceca-a85c-4eda-8e21-7e64c0cc08b2 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:45.260989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3ecdceca-a85c-4eda-8e21-7e64c0cc08b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:45.268709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 729/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:45.428005 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337e8c6-a020-4812-9215-c0a0fbfcc3f9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:45.442806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337e8c6-a020-4812-9215-c0a0fbfcc3f9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:45.448118 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eef611-1289-40e9-a8e7-72c853a2d375 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:45.448755 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eef611-1289-40e9-a8e7-72c853a2d375 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:45.449039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eef611-1289-40e9-a8e7-72c853a2d375 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:45.449319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eef611-1289-40e9-a8e7-72c853a2d375 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:45.457518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eef611-1289-40e9-a8e7-72c853a2d375 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:45.459120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eef611-1289-40e9-a8e7-72c853a2d375 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:45.468246 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 702/1431] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:46.330569 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7754d-c76c-48e1-ad11-d6acb0553d82 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:46.331188 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7754d-c76c-48e1-ad11-d6acb0553d82 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:46.331188 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7754d-c76c-48e1-ad11-d6acb0553d82 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:46.333065 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 730/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:29:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:46.339337 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdbb0e-b172-4df1-b4e7-b3c5d65be5bd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:46.339827 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdbb0e-b172-4df1-b4e7-b3c5d65be5bd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:46.340011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdbb0e-b172-4df1-b4e7-b3c5d65be5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:46.359252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-30cdbb0e-b172-4df1-b4e7-b3c5d65be5bd None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:46.360488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-30cdbb0e-b172-4df1-b4e7-b3c5d65be5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:46.368677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 703/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:46.516241 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7754d-c76c-48e1-ad11-d6acb0553d82 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:46.531578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7754d-c76c-48e1-ad11-d6acb0553d82 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:46.537545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d83bd-c8c7-4099-aae8-04e2faabb087 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:46.538196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d83bd-c8c7-4099-aae8-04e2faabb087 service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:46.538485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d83bd-c8c7-4099-aae8-04e2faabb087 service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:46.538770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d83bd-c8c7-4099-aae8-04e2faabb087 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:46.547603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d83bd-c8c7-4099-aae8-04e2faabb087 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:46.549195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d83bd-c8c7-4099-aae8-04e2faabb087 service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:46.559326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 731/1434] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:47.363307 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22705cd-a20e-47a6-8f79-c154f0c8236d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:47.363881 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22705cd-a20e-47a6-8f79-c154f0c8236d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:47.363881 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22705cd-a20e-47a6-8f79-c154f0c8236d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:47.365206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 704/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:29:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:47.370053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6d4d3-1d81-4447-a275-3c568512f88b None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:47.370485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6d4d3-1d81-4447-a275-3c568512f88b None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:47.370610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6d4d3-1d81-4447-a275-3c568512f88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:47.388326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-bed6d4d3-1d81-4447-a275-3c568512f88b None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:47.389548 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-bed6d4d3-1d81-4447-a275-3c568512f88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:47.397468 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 732/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:29:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:47.552027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22705cd-a20e-47a6-8f79-c154f0c8236d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:47.566040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22705cd-a20e-47a6-8f79-c154f0c8236d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:47.571306 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ef47e-f94e-43ae-9e6e-e6bc10482864 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:47.571937 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ef47e-f94e-43ae-9e6e-e6bc10482864 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:47.572223 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ef47e-f94e-43ae-9e6e-e6bc10482864 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:47.572501 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ef47e-f94e-43ae-9e6e-e6bc10482864 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:47.580548 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ef47e-f94e-43ae-9e6e-e6bc10482864 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:47.582509 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ef47e-f94e-43ae-9e6e-e6bc10482864 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:47.591672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 705/1437] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:48.402315 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09cd0d-870e-42c5-8b15-e5b81a1fc184 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:48.402862 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09cd0d-870e-42c5-8b15-e5b81a1fc184 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:48.402862 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09cd0d-870e-42c5-8b15-e5b81a1fc184 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:48.404468 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 733/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:48.409446 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5361df7-ca62-4f80-962e-850da5b4a8ff None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:48.409860 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5361df7-ca62-4f80-962e-850da5b4a8ff None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:48.410154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5361df7-ca62-4f80-962e-850da5b4a8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:48.428526 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d5361df7-ca62-4f80-962e-850da5b4a8ff None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:48.429703 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d5361df7-ca62-4f80-962e-850da5b4a8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:48.437326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 706/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:48.588442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09cd0d-870e-42c5-8b15-e5b81a1fc184 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:48.602499 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09cd0d-870e-42c5-8b15-e5b81a1fc184 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:48.607902 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facad4f-2d83-4840-9ade-316eb1b574af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:48.608541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facad4f-2d83-4840-9ade-316eb1b574af service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:48.608833 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facad4f-2d83-4840-9ade-316eb1b574af service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:48.609118 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facad4f-2d83-4840-9ade-316eb1b574af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:48.617681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facad4f-2d83-4840-9ade-316eb1b574af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:48.619295 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facad4f-2d83-4840-9ade-316eb1b574af service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:48.629243 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 734/1440] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:50.563311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419e3eb-b8ca-468d-97c3-1bbb650b0582 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:50.563908 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419e3eb-b8ca-468d-97c3-1bbb650b0582 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:50.564374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a419e3eb-b8ca-468d-97c3-1bbb650b0582 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:50.565770 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 707/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:29:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:50.571115 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dfb03-4462-4eff-82f7-49c51919f5c9 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:50.571520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dfb03-4462-4eff-82f7-49c51919f5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:50.571684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200dfb03-4462-4eff-82f7-49c51919f5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:50.589466 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-200dfb03-4462-4eff-82f7-49c51919f5c9 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:50.590659 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-200dfb03-4462-4eff-82f7-49c51919f5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:50.599111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 735/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:50.637293 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419e3eb-b8ca-468d-97c3-1bbb650b0582 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:50.647986 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a419e3eb-b8ca-468d-97c3-1bbb650b0582 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:50.651447 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250df9f9-0e10-4345-9d89-1e95a22ba7bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:50.652079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250df9f9-0e10-4345-9d89-1e95a22ba7bf service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:50.652368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250df9f9-0e10-4345-9d89-1e95a22ba7bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:50.652648 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250df9f9-0e10-4345-9d89-1e95a22ba7bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:50.662558 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250df9f9-0e10-4345-9d89-1e95a22ba7bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:50.664008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250df9f9-0e10-4345-9d89-1e95a22ba7bf service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:50.673259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 708/1443] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:52.863307 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b188c-6a27-4b10-802b-03b0d3a1889d None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:52.863307 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b188c-6a27-4b10-802b-03b0d3a1889d None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:52.864416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b188c-6a27-4b10-802b-03b0d3a1889d None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:52.864845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 736/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:29:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:52.870643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2aac7-4881-4684-a68e-499307a3ed62 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:52.871317 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2aac7-4881-4684-a68e-499307a3ed62 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:52.871317 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2aac7-4881-4684-a68e-499307a3ed62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:52.889636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-08a2aac7-4881-4684-a68e-499307a3ed62 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:52.890858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-08a2aac7-4881-4684-a68e-499307a3ed62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:52.900163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 709/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:29:52.940137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b188c-6a27-4b10-802b-03b0d3a1889d None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:52.950657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b188c-6a27-4b10-802b-03b0d3a1889d None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:52.954149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2278e-f777-4358-b301-89d9aa16c4bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:52.954789 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2278e-f777-4358-b301-89d9aa16c4bc service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:52.955116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2278e-f777-4358-b301-89d9aa16c4bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:52.955403 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2278e-f777-4358-b301-89d9aa16c4bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:52.964988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2278e-f777-4358-b301-89d9aa16c4bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:52.966417 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2278e-f777-4358-b301-89d9aa16c4bc service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:52.976554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 737/1446] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:55.337654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01491a-1a08-4f06-9d43-dbd651426ef8 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:55.338211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01491a-1a08-4f06-9d43-dbd651426ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:55.338211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01491a-1a08-4f06-9d43-dbd651426ef8 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:55.339571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 710/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:55.344768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e3eb8-f867-42ac-ae79-d89c7dc29ff3 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:55.345167 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e3eb8-f867-42ac-ae79-d89c7dc29ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:55.345338 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e3eb8-f867-42ac-ae79-d89c7dc29ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:55.363382 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-9a0e3eb8-f867-42ac-ae79-d89c7dc29ff3 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:55.364575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-9a0e3eb8-f867-42ac-ae79-d89c7dc29ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:55.372390 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 738/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:55.411690 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01491a-1a08-4f06-9d43-dbd651426ef8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:55.422145 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01491a-1a08-4f06-9d43-dbd651426ef8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:55.425624 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75414177-8c27-4287-b3e1-b8b35c84349f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:55.426250 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75414177-8c27-4287-b3e1-b8b35c84349f service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:55.426531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75414177-8c27-4287-b3e1-b8b35c84349f service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:55.426806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75414177-8c27-4287-b3e1-b8b35c84349f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:55.435625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75414177-8c27-4287-b3e1-b8b35c84349f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:55.437033 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75414177-8c27-4287-b3e1-b8b35c84349f service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:55.446053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 711/1449] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:57.622689 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42391166-0845-494f-a778-b765f19a5d25 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:57.623289 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42391166-0845-494f-a778-b765f19a5d25 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:57.623289 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42391166-0845-494f-a778-b765f19a5d25 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:57.624690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 739/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:57.630453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41575dd-62d1-45a8-8a35-1a7014797b29 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:57.630891 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41575dd-62d1-45a8-8a35-1a7014797b29 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:57.631090 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41575dd-62d1-45a8-8a35-1a7014797b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:57.649061 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b41575dd-62d1-45a8-8a35-1a7014797b29 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:57.650235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b41575dd-62d1-45a8-8a35-1a7014797b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:57.658112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 712/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:29:57.696233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42391166-0845-494f-a778-b765f19a5d25 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:57.707477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42391166-0845-494f-a778-b765f19a5d25 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:57.710864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfee3b2-6d46-4c79-895e-c3a51ff13bde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:57.711532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfee3b2-6d46-4c79-895e-c3a51ff13bde service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:57.711841 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfee3b2-6d46-4c79-895e-c3a51ff13bde service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:57.712120 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfee3b2-6d46-4c79-895e-c3a51ff13bde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:57.720658 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfee3b2-6d46-4c79-895e-c3a51ff13bde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:57.722408 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfee3b2-6d46-4c79-895e-c3a51ff13bde service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:57.731546 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 740/1452] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:29:59.847720 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af164d-f1ac-4c7e-8431-7a2a17768240 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:59.848267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af164d-f1ac-4c7e-8431-7a2a17768240 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:59.848267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af164d-f1ac-4c7e-8431-7a2a17768240 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:59.849641 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 713/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:29:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:29:59.854882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f080d-5f45-4628-9555-011d592b2c1f None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:59.855335 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f080d-5f45-4628-9555-011d592b2c1f None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:59.855491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f080d-5f45-4628-9555-011d592b2c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:59.874064 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-a06f080d-5f45-4628-9555-011d592b2c1f None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:29:59.875301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-a06f080d-5f45-4628-9555-011d592b2c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:29:59.883622 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 741/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:29:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:29:59.922764 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af164d-f1ac-4c7e-8431-7a2a17768240 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:29:59.934541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af164d-f1ac-4c7e-8431-7a2a17768240 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:29:59.938053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5cc1b-2be2-4f0d-b03b-fa18cfa85690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:29:59.938688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5cc1b-2be2-4f0d-b03b-fa18cfa85690 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:29:59.939004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5cc1b-2be2-4f0d-b03b-fa18cfa85690 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:29:59.939290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5cc1b-2be2-4f0d-b03b-fa18cfa85690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:29:59.948555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5cc1b-2be2-4f0d-b03b-fa18cfa85690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:29:59.949970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5cc1b-2be2-4f0d-b03b-fa18cfa85690 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:29:59.959873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 714/1455] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:29:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:30:02.105596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcada30-bc47-4167-92c1-8f25848a810c None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:02.106168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcada30-bc47-4167-92c1-8f25848a810c None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:02.106168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcada30-bc47-4167-92c1-8f25848a810c None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:02.107567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 742/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:02.113068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ca97c-67e1-4e1d-a285-c5920814705c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:02.113503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ca97c-67e1-4e1d-a285-c5920814705c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:02.113658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ca97c-67e1-4e1d-a285-c5920814705c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:02.133251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ae4ca97c-67e1-4e1d-a285-c5920814705c None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:02.134538 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ae4ca97c-67e1-4e1d-a285-c5920814705c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:02.142894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 715/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:30:02.181932 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcada30-bc47-4167-92c1-8f25848a810c None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:02.194074 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcada30-bc47-4167-92c1-8f25848a810c None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:02.197780 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c1475-f1dd-4a61-ac54-bfda974b8cce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:02.198396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c1475-f1dd-4a61-ac54-bfda974b8cce service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:02.198689 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c1475-f1dd-4a61-ac54-bfda974b8cce service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:02.199069 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c1475-f1dd-4a61-ac54-bfda974b8cce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:02.209295 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c1475-f1dd-4a61-ac54-bfda974b8cce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:02.210734 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c1475-f1dd-4a61-ac54-bfda974b8cce service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:02.220543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 743/1458] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:30:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:30:04.302767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d238fa-8d4d-49b0-9fb9-085a424ff74d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:04.303348 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d238fa-8d4d-49b0-9fb9-085a424ff74d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:04.303348 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d238fa-8d4d-49b0-9fb9-085a424ff74d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:04.304741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 716/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:04.309906 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9bde3-9fcf-49bb-b0a1-917ffd49aa42 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:04.310319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9bde3-9fcf-49bb-b0a1-917ffd49aa42 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:04.310496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9bde3-9fcf-49bb-b0a1-917ffd49aa42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:04.327949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-4fa9bde3-9fcf-49bb-b0a1-917ffd49aa42 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:04.329129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-4fa9bde3-9fcf-49bb-b0a1-917ffd49aa42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:04.336860 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 744/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:30:04.375252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d238fa-8d4d-49b0-9fb9-085a424ff74d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:04.388905 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d238fa-8d4d-49b0-9fb9-085a424ff74d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:04.391583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6453b-6968-4997-9677-3e9cf4f0c889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:04.392222 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6453b-6968-4997-9677-3e9cf4f0c889 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:04.392508 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6453b-6968-4997-9677-3e9cf4f0c889 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:04.392793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6453b-6968-4997-9677-3e9cf4f0c889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:04.401833 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6453b-6968-4997-9677-3e9cf4f0c889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:04.403548 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6453b-6968-4997-9677-3e9cf4f0c889 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:04.414108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 717/1461] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:30:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:30:06.505974 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb39bd6-452c-427e-b389-b0a9cd6d27d1 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:06.506550 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb39bd6-452c-427e-b389-b0a9cd6d27d1 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:06.506550 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb39bd6-452c-427e-b389-b0a9cd6d27d1 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:06.507979 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 745/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:06.513168 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6543e-be0c-4b5a-93e6-3b214e402fb9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:06.513589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6543e-be0c-4b5a-93e6-3b214e402fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:06.513728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6543e-be0c-4b5a-93e6-3b214e402fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:06.520410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=79 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:06.533332 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4ec6543e-be0c-4b5a-93e6-3b214e402fb9 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:06.534564 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4ec6543e-be0c-4b5a-93e6-3b214e402fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:06.542219 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 718/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:30:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:30:06.579206 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb39bd6-452c-427e-b389-b0a9cd6d27d1 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:06.585568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=73 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:06.592046 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb39bd6-452c-427e-b389-b0a9cd6d27d1 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:06.595551 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5585ce-751b-49dd-a03c-0712ff341624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:06.596372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5585ce-751b-49dd-a03c-0712ff341624 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:06.596559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5585ce-751b-49dd-a03c-0712ff341624 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:06.596845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5585ce-751b-49dd-a03c-0712ff341624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:06.605630 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5585ce-751b-49dd-a03c-0712ff341624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:06.607101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5585ce-751b-49dd-a03c-0712ff341624 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:06.616364 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 746/1464] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:30:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:30:08.676784 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295c67d-195c-49c3-be2e-2febc90e91af None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:08.677346 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295c67d-195c-49c3-be2e-2febc90e91af None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:08.677346 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295c67d-195c-49c3-be2e-2febc90e91af None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:08.678710 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 719/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:08.683955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a3699-6f6c-497a-8854-a41751490ecb None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:08.684322 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a3699-6f6c-497a-8854-a41751490ecb None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:08.684518 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a3699-6f6c-497a-8854-a41751490ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:08.702261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-dd2a3699-6f6c-497a-8854-a41751490ecb None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:08.703461 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-dd2a3699-6f6c-497a-8854-a41751490ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:08.711111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 747/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:30:08.746907 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295c67d-195c-49c3-be2e-2febc90e91af None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:08.758259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295c67d-195c-49c3-be2e-2febc90e91af None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:08.761695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61429a-3e37-4504-bd53-76a42adaf531 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:08.762327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61429a-3e37-4504-bd53-76a42adaf531 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:08.762612 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61429a-3e37-4504-bd53-76a42adaf531 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:08.762890 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61429a-3e37-4504-bd53-76a42adaf531 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:08.771836 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61429a-3e37-4504-bd53-76a42adaf531 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:08.773249 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61429a-3e37-4504-bd53-76a42adaf531 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:08.782618 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 720/1467] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:30:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:30:10.851861 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7818b0-edb4-4680-8982-b9ee9573ebd8 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:10.852385 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7818b0-edb4-4680-8982-b9ee9573ebd8 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:10.852512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7818b0-edb4-4680-8982-b9ee9573ebd8 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:10.853785 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 748/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:10.859104 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d7f93-856f-4fbe-975e-b4fb7c6d589e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:10.859554 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d7f93-856f-4fbe-975e-b4fb7c6d589e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:10.859682 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d7f93-856f-4fbe-975e-b4fb7c6d589e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:10.877410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-632d7f93-856f-4fbe-975e-b4fb7c6d589e None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:10.878742 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-632d7f93-856f-4fbe-975e-b4fb7c6d589e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:10.886674 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 721/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:30:10.924940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7818b0-edb4-4680-8982-b9ee9573ebd8 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:10.935911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7818b0-edb4-4680-8982-b9ee9573ebd8 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:10.939387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151b91e-8fee-4d6a-867b-55b7b1cbec46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:10.940218 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151b91e-8fee-4d6a-867b-55b7b1cbec46 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:10.940520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151b91e-8fee-4d6a-867b-55b7b1cbec46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:10.940806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151b91e-8fee-4d6a-867b-55b7b1cbec46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:10.950142 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151b91e-8fee-4d6a-867b-55b7b1cbec46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:10.951624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151b91e-8fee-4d6a-867b-55b7b1cbec46 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:10.961617 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 749/1470] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:30:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:30:20.872471 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=11 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:20.957386 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=14 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:31.019068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489bb11-534e-4611-92cc-064ee3ef50c0 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:31.019619 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489bb11-534e-4611-92cc-064ee3ef50c0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:31.019619 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489bb11-534e-4611-92cc-064ee3ef50c0 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:31.021043 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 722/1471] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:31.026681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed632f8-bda9-4d37-a773-61c038c16c96 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:31.027186 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed632f8-bda9-4d37-a773-61c038c16c96 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:31.027293 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed632f8-bda9-4d37-a773-61c038c16c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:31.045875 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-4ed632f8-bda9-4d37-a773-61c038c16c96 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:31.047141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-4ed632f8-bda9-4d37-a773-61c038c16c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:31.056329 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 750/1472] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:30:31.110060 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e90be0-e745-4f5a-8db3-f4d3bd91d650 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:31.110361 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e90be0-e745-4f5a-8db3-f4d3bd91d650 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:31.110653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e90be0-e745-4f5a-8db3-f4d3bd91d650 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:31.112611 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 723/1473] 10.222.0.43 () {54 vars in 962 bytes} [Thu Jun 23 22:30:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:31.117616 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b329b98-e6aa-433c-b4d3-c1a37f2fcdc9 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:31.117919 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b329b98-e6aa-433c-b4d3-c1a37f2fcdc9 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:31.118210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b329b98-e6aa-433c-b4d3-c1a37f2fcdc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:31.142334 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-2b329b98-e6aa-433c-b4d3-c1a37f2fcdc9 None None] MFA Rules not processed for user `36de21e0b3de48dabe0f27a9db8a37aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:31.143861 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2b329b98-e6aa-433c-b4d3-c1a37f2fcdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:31.150461 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 751/1474] 10.222.0.43 () {58 vars in 1059 bytes} [Thu Jun 23 22:30:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:30:31.158275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e90be0-e745-4f5a-8db3-f4d3bd91d650 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:31.169223 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e90be0-e745-4f5a-8db3-f4d3bd91d650 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:31.175459 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b706f-c076-4dda-8918-943eda067553 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36de21e0b3de48dabe0f27a9db8a37aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:31.176116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b706f-c076-4dda-8918-943eda067553 service glance] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:31.176401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b706f-c076-4dda-8918-943eda067553 service glance] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:31.176686 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b706f-c076-4dda-8918-943eda067553 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:31.185936 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b706f-c076-4dda-8918-943eda067553 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:31.187406 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b706f-c076-4dda-8918-943eda067553 service glance] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:31.198833 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 724/1475] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:30:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:30:33.371676 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d71ad-057a-48eb-8477-ae0314a22d23 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:33.372220 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d71ad-057a-48eb-8477-ae0314a22d23 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:33.372220 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d71ad-057a-48eb-8477-ae0314a22d23 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:33.373660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 752/1476] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:33.378986 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40a436-9476-4df3-847e-39a45fe92b50 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:33.379421 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40a436-9476-4df3-847e-39a45fe92b50 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:33.379622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40a436-9476-4df3-847e-39a45fe92b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:33.398484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ad40a436-9476-4df3-847e-39a45fe92b50 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:33.399814 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ad40a436-9476-4df3-847e-39a45fe92b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:33.409093 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 725/1477] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:30:33.432503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d71ad-057a-48eb-8477-ae0314a22d23 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:33.445712 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d71ad-057a-48eb-8477-ae0314a22d23 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:33.450274 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1953f-7446-4d68-b984-182f1f0ca021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:33.450996 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1953f-7446-4d68-b984-182f1f0ca021 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:33.451285 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1953f-7446-4d68-b984-182f1f0ca021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:33.451570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1953f-7446-4d68-b984-182f1f0ca021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:33.452316 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1953f-7446-4d68-b984-182f1f0ca021 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:33.454105 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: WARNING py.warnings [None req-08e1953f-7446-4d68-b984-182f1f0ca021 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:30:33.454105 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: warnings.warn(msg) Jun 23 22:30:33.454105 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: Jun 23 22:30:33.455887 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1953f-7446-4d68-b984-182f1f0ca021 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:33.476406 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 753/1478] 10.222.0.43 () {56 vars in 1108 bytes} [Thu Jun 23 22:30:33 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:30:35.583350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ec7a4-0a07-45bf-97c9-82b63e8576b9 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:35.583889 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ec7a4-0a07-45bf-97c9-82b63e8576b9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:35.583989 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ec7a4-0a07-45bf-97c9-82b63e8576b9 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:35.585293 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 726/1479] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:35.590668 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672f377-22d0-4fc9-a9f5-976e996877fb None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:35.591116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672f377-22d0-4fc9-a9f5-976e996877fb None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:35.591296 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672f377-22d0-4fc9-a9f5-976e996877fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:35.612405 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-7672f377-22d0-4fc9-a9f5-976e996877fb None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:35.613614 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7672f377-22d0-4fc9-a9f5-976e996877fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:35.621305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 754/1480] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:30:35.664904 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ec7a4-0a07-45bf-97c9-82b63e8576b9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:35.679809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ec7a4-0a07-45bf-97c9-82b63e8576b9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:35.685152 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53494f9c-542b-4108-92f0-fe912bf8152e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:35.685799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53494f9c-542b-4108-92f0-fe912bf8152e service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:35.686089 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53494f9c-542b-4108-92f0-fe912bf8152e service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:35.686373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53494f9c-542b-4108-92f0-fe912bf8152e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:35.695209 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53494f9c-542b-4108-92f0-fe912bf8152e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:35.696796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53494f9c-542b-4108-92f0-fe912bf8152e service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:35.706541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 727/1481] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:30:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:30:37.913013 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5914ee-2226-4ac1-85e7-f20d4aa0b3b5 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:37.913576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5914ee-2226-4ac1-85e7-f20d4aa0b3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:37.913576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5914ee-2226-4ac1-85e7-f20d4aa0b3b5 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:37.914936 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 755/1482] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:37.920089 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21f42a-5d60-4007-98a4-365f35a1f721 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:37.920492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21f42a-5d60-4007-98a4-365f35a1f721 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:37.920660 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21f42a-5d60-4007-98a4-365f35a1f721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:37.938652 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5e21f42a-5d60-4007-98a4-365f35a1f721 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:37.940659 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5e21f42a-5d60-4007-98a4-365f35a1f721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:37.948391 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 728/1483] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:30:37.985348 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5914ee-2226-4ac1-85e7-f20d4aa0b3b5 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:37.997205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5914ee-2226-4ac1-85e7-f20d4aa0b3b5 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:38.000708 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b27a8-2f62-44c6-aacf-826a4ca1b48c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:38.001376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b27a8-2f62-44c6-aacf-826a4ca1b48c service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:38.002238 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b27a8-2f62-44c6-aacf-826a4ca1b48c service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:38.002516 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b27a8-2f62-44c6-aacf-826a4ca1b48c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:38.011389 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b27a8-2f62-44c6-aacf-826a4ca1b48c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:38.012824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b27a8-2f62-44c6-aacf-826a4ca1b48c service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:38.022255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 756/1484] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:30:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:30:40.152239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f7d7e-27be-44b0-a8ea-835e2c361d6a None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:40.152750 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f7d7e-27be-44b0-a8ea-835e2c361d6a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:40.152854 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f7d7e-27be-44b0-a8ea-835e2c361d6a None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:40.154502 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 729/1485] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:40.160574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81158b42-5e89-4d6a-a0d7-3425460e7ff6 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:40.161036 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81158b42-5e89-4d6a-a0d7-3425460e7ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:40.161149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81158b42-5e89-4d6a-a0d7-3425460e7ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:40.178455 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-81158b42-5e89-4d6a-a0d7-3425460e7ff6 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:40.179728 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-81158b42-5e89-4d6a-a0d7-3425460e7ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:40.187395 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 757/1486] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:30:40.226732 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f7d7e-27be-44b0-a8ea-835e2c361d6a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:40.237304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f7d7e-27be-44b0-a8ea-835e2c361d6a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:40.240877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3806c-0342-4703-946d-a8f3598e92d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:40.241511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3806c-0342-4703-946d-a8f3598e92d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:40.241796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3806c-0342-4703-946d-a8f3598e92d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:40.242076 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3806c-0342-4703-946d-a8f3598e92d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:40.251022 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3806c-0342-4703-946d-a8f3598e92d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:40.252976 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3806c-0342-4703-946d-a8f3598e92d4 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:40.263308 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 730/1487] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:30:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:30:42.449056 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4b20a-b0d7-4bc5-a277-584a5af2b01f None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:42.449576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4b20a-b0d7-4bc5-a277-584a5af2b01f None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:42.449576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4b20a-b0d7-4bc5-a277-584a5af2b01f None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:42.450892 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 758/1488] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:42.456653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b2a12-03d9-498e-964f-63d65c114141 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:42.457071 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b2a12-03d9-498e-964f-63d65c114141 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:42.457212 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b2a12-03d9-498e-964f-63d65c114141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:42.475179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b54b2a12-03d9-498e-964f-63d65c114141 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:42.476372 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b54b2a12-03d9-498e-964f-63d65c114141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:42.484000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 731/1489] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:30:42.524901 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4b20a-b0d7-4bc5-a277-584a5af2b01f None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:42.536032 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4b20a-b0d7-4bc5-a277-584a5af2b01f None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:42.539467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6841be-1cf2-4915-9fbb-3887978597d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:42.540111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6841be-1cf2-4915-9fbb-3887978597d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:42.540394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6841be-1cf2-4915-9fbb-3887978597d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:42.540673 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6841be-1cf2-4915-9fbb-3887978597d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:42.549390 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6841be-1cf2-4915-9fbb-3887978597d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:42.550807 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6841be-1cf2-4915-9fbb-3887978597d6 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:42.560029 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 759/1490] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:30:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:30:44.789108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e1bc0-433a-446d-b91a-bbc167738298 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:44.789650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e1bc0-433a-446d-b91a-bbc167738298 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:44.789650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e1bc0-433a-446d-b91a-bbc167738298 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:44.791062 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 732/1491] 10.222.0.43 () {54 vars in 926 bytes} [Thu Jun 23 22:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:30:44.796333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93737e1f-b078-4658-acdf-d9aa6bf394ac None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:44.796752 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93737e1f-b078-4658-acdf-d9aa6bf394ac None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:44.796899 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93737e1f-b078-4658-acdf-d9aa6bf394ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:44.814673 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-93737e1f-b078-4658-acdf-d9aa6bf394ac None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:30:44.815883 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-93737e1f-b078-4658-acdf-d9aa6bf394ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:30:44.823499 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 760/1492] 10.222.0.43 () {58 vars in 1023 bytes} [Thu Jun 23 22:30:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:30:44.862457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e1bc0-433a-446d-b91a-bbc167738298 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:30:44.873264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e1bc0-433a-446d-b91a-bbc167738298 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:30:44.877245 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c96348-39e9-478c-be5d-6be61c2d6c71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:30:44.877875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c96348-39e9-478c-be5d-6be61c2d6c71 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:30:44.878160 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c96348-39e9-478c-be5d-6be61c2d6c71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:30:44.878440 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c96348-39e9-478c-be5d-6be61c2d6c71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:30:44.887170 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c96348-39e9-478c-be5d-6be61c2d6c71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:30:44.888578 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c96348-39e9-478c-be5d-6be61c2d6c71 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:30:44.898503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 733/1493] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:30:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:30:54.809246 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=33 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:30:54.894430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=33 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:32:10.954469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54502a-bf30-4574-8895-89ba8f7ea100 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:32:10.955093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54502a-bf30-4574-8895-89ba8f7ea100 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:32:10.955093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54502a-bf30-4574-8895-89ba8f7ea100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:32:10.973434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-2e54502a-bf30-4574-8895-89ba8f7ea100 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:32:10.974635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2e54502a-bf30-4574-8895-89ba8f7ea100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:32:10.982627 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 761/1494] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:32:10.992044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3154d7-8e8f-41c3-9270-1600c7cd3396 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:32:10.992532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3154d7-8e8f-41c3-9270-1600c7cd3396 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:32:10.992639 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3154d7-8e8f-41c3-9270-1600c7cd3396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:32:11.010708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6a3154d7-8e8f-41c3-9270-1600c7cd3396 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:32:11.011862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6a3154d7-8e8f-41c3-9270-1600c7cd3396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:32:11.020272 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 734/1495] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:32:11.024931 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54502a-bf30-4574-8895-89ba8f7ea100 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:32:11.035556 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54502a-bf30-4574-8895-89ba8f7ea100 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:32:11.040107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a29ef0-4725-4717-9e48-67552259d959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:32:11.040773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a29ef0-4725-4717-9e48-67552259d959 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:32:11.041063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a29ef0-4725-4717-9e48-67552259d959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:32:11.041347 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a29ef0-4725-4717-9e48-67552259d959 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:32:11.042080 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a29ef0-4725-4717-9e48-67552259d959 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:32:11.042394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a29ef0-4725-4717-9e48-67552259d959 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:32:11.043458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: WARNING py.warnings [None req-75a29ef0-4725-4717-9e48-67552259d959 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:32:11.043458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: warnings.warn(msg) Jun 23 22:32:11.043458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: Jun 23 22:32:11.044348 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a29ef0-4725-4717-9e48-67552259d959 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:32:11.056125 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 762/1496] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:32:11.064735 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3154d7-8e8f-41c3-9270-1600c7cd3396 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:32:11.075170 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3154d7-8e8f-41c3-9270-1600c7cd3396 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:32:11.081283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab06f95-3d35-4aa9-9c2b-860d5873a2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:32:11.082010 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab06f95-3d35-4aa9-9c2b-860d5873a2bb admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:32:11.082299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab06f95-3d35-4aa9-9c2b-860d5873a2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:32:11.082584 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab06f95-3d35-4aa9-9c2b-860d5873a2bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:32:11.083329 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab06f95-3d35-4aa9-9c2b-860d5873a2bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:32:11.085091 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-3ab06f95-3d35-4aa9-9c2b-860d5873a2bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:32:11.085091 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Jun 23 22:32:11.085091 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: Jun 23 22:32:11.086375 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab06f95-3d35-4aa9-9c2b-860d5873a2bb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:32:11.121900 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 735/1497] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:32:11 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:32:11.127532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a29ef0-4725-4717-9e48-67552259d959 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:32:11.138369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a29ef0-4725-4717-9e48-67552259d959 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:32:11.194771 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6086ed-e0be-4c2d-9c82-f1482d22de96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:32:11.195524 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6086ed-e0be-4c2d-9c82-f1482d22de96 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:32:11.195839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6086ed-e0be-4c2d-9c82-f1482d22de96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:32:11.196129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6086ed-e0be-4c2d-9c82-f1482d22de96 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:32:11.196815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6086ed-e0be-4c2d-9c82-f1482d22de96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:32:11.197829 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: WARNING py.warnings [None req-5a6086ed-e0be-4c2d-9c82-f1482d22de96 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:32:11.197829 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: warnings.warn(msg) Jun 23 22:32:11.197829 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: Jun 23 22:32:11.199094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6086ed-e0be-4c2d-9c82-f1482d22de96 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:32:11.221029 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 763/1498] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:32:11 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:32:11.225226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab06f95-3d35-4aa9-9c2b-860d5873a2bb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:32:11.236090 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab06f95-3d35-4aa9-9c2b-860d5873a2bb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:32:11.241064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c1400-0b50-4c09-9104-c949556232dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:32:11.241773 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c1400-0b50-4c09-9104-c949556232dd admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:32:11.242060 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c1400-0b50-4c09-9104-c949556232dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:32:11.242340 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c1400-0b50-4c09-9104-c949556232dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:32:11.243121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c1400-0b50-4c09-9104-c949556232dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:32:11.244128 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-ff1c1400-0b50-4c09-9104-c949556232dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:32:11.244128 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Jun 23 22:32:11.244128 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: Jun 23 22:32:11.245002 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c1400-0b50-4c09-9104-c949556232dd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:32:11.257795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 736/1499] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:32:11.262144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6086ed-e0be-4c2d-9c82-f1482d22de96 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:32:11.272562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6086ed-e0be-4c2d-9c82-f1482d22de96 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:32:11.277101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae257e-ca89-4b9b-9eec-36a4acada360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:32:11.277759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae257e-ca89-4b9b-9eec-36a4acada360 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:32:11.278049 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae257e-ca89-4b9b-9eec-36a4acada360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:32:11.278336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae257e-ca89-4b9b-9eec-36a4acada360 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:32:11.279102 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae257e-ca89-4b9b-9eec-36a4acada360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:32:11.280367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: WARNING py.warnings [None req-1eae257e-ca89-4b9b-9eec-36a4acada360 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:32:11.280367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: warnings.warn(msg) Jun 23 22:32:11.280367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: Jun 23 22:32:11.282348 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae257e-ca89-4b9b-9eec-36a4acada360 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:32:11.295547 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 764/1500] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:32:11.300365 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c1400-0b50-4c09-9104-c949556232dd admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:32:11.310017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c1400-0b50-4c09-9104-c949556232dd admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:32:11.314689 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377478c-5ae3-4391-9159-4894ea53a246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:32:11.315599 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377478c-5ae3-4391-9159-4894ea53a246 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:32:11.315886 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377478c-5ae3-4391-9159-4894ea53a246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:32:11.316190 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377478c-5ae3-4391-9159-4894ea53a246 admin admin] PATH_INFO: `/v3/projects/fb06b0948d804de086aca1cfddea1454/users/97793a227d354a99a9f82d99f3ffd93d/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:32:11.331659 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377478c-5ae3-4391-9159-4894ea53a246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb06b0948d804de086aca1cfddea1454, user_id=97793a227d354a99a9f82d99f3ffd93d, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:32:11.332674 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-5377478c-5ae3-4391-9159-4894ea53a246 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:32:11.332674 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Jun 23 22:32:11.332674 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: Jun 23 22:32:11.334590 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377478c-5ae3-4391-9159-4894ea53a246 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:32:11.347964 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 737/1501] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:32:11 2022] PUT /identity/v3/projects/fb06b0948d804de086aca1cfddea1454/users/97793a227d354a99a9f82d99f3ffd93d/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:32:11.359333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862de7c-d550-4ed6-8a81-78a317c2a6c1 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:32:11.359706 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862de7c-d550-4ed6-8a81-78a317c2a6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:32:11.360027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862de7c-d550-4ed6-8a81-78a317c2a6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:32:11.386687 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-b862de7c-d550-4ed6-8a81-78a317c2a6c1 None None] MFA Rules not processed for user `97793a227d354a99a9f82d99f3ffd93d`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:32:11.387976 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b862de7c-d550-4ed6-8a81-78a317c2a6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:32:11.452243 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 765/1502] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:32:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:32:11.465517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bad7c-fc87-4eb9-ac27-cc2a493aada1 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:32:11.465831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bad7c-fc87-4eb9-ac27-cc2a493aada1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:32:11.466243 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bad7c-fc87-4eb9-ac27-cc2a493aada1 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:32:11.467738 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 738/1503] 10.222.0.43 () {54 vars in 963 bytes} [Thu Jun 23 22:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:32:11.472641 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c68ea5-771f-44a0-8db6-01457173dbea None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:32:11.472944 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c68ea5-771f-44a0-8db6-01457173dbea None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:32:11.473235 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c68ea5-771f-44a0-8db6-01457173dbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:32:11.491795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-63c68ea5-771f-44a0-8db6-01457173dbea None None] MFA Rules not processed for user `95bc067d93fa4207bec1c85600728085`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:32:11.493004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-63c68ea5-771f-44a0-8db6-01457173dbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:32:11.539570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:32:11.548529 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 766/1504] 10.222.0.43 () {58 vars in 1060 bytes} [Thu Jun 23 22:32:11 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:32:11.556386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9bad7c-fc87-4eb9-ac27-cc2a493aada1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:32:11.567850 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9bad7c-fc87-4eb9-ac27-cc2a493aada1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:32:11.571525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017a623-a8c7-4c59-a394-50a6bf4c257a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:32:11.572178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017a623-a8c7-4c59-a394-50a6bf4c257a service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:32:11.572474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017a623-a8c7-4c59-a394-50a6bf4c257a service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:32:11.572756 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017a623-a8c7-4c59-a394-50a6bf4c257a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:32:11.582528 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7017a623-a8c7-4c59-a394-50a6bf4c257a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:32:11.584270 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7017a623-a8c7-4c59-a394-50a6bf4c257a service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:32:11.596251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 739/1505] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:32:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:32:11.639956 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11faad-d435-4170-b7bd-a0cb7c5b60fa None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:32:11.640459 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11faad-d435-4170-b7bd-a0cb7c5b60fa None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:32:11.640753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11faad-d435-4170-b7bd-a0cb7c5b60fa None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:32:11.642034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 767/1506] 10.222.0.43 () {52 vars in 829 bytes} [Thu Jun 23 22:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:32:11.645640 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017a623-a8c7-4c59-a394-50a6bf4c257a service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:32:11.827272 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017a623-a8c7-4c59-a394-50a6bf4c257a service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:32:11.833630 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d409e0-83b8-450e-bfd3-7b1baa912305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:32:11.834557 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d409e0-83b8-450e-bfd3-7b1baa912305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:32:11.834845 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d409e0-83b8-450e-bfd3-7b1baa912305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:32:11.835197 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d409e0-83b8-450e-bfd3-7b1baa912305 admin admin] PATH_INFO: `/v3/users/97793a227d354a99a9f82d99f3ffd93d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:32:11.836388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d409e0-83b8-450e-bfd3-7b1baa912305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97793a227d354a99a9f82d99f3ffd93d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:32:11.837385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-50d409e0-83b8-450e-bfd3-7b1baa912305 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:32:11.837385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Jun 23 22:32:11.837385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: Jun 23 22:32:11.838604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d409e0-83b8-450e-bfd3-7b1baa912305 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:32:11.903044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-50d409e0-83b8-450e-bfd3-7b1baa912305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97793a227d354a99a9f82d99f3ffd93d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:32:11.916262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-50d409e0-83b8-450e-bfd3-7b1baa912305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97793a227d354a99a9f82d99f3ffd93d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:32:11.928338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-50d409e0-83b8-450e-bfd3-7b1baa912305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97793a227d354a99a9f82d99f3ffd93d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:32:11.929269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-50d409e0-83b8-450e-bfd3-7b1baa912305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97793a227d354a99a9f82d99f3ffd93d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:32:11.929930 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-50d409e0-83b8-450e-bfd3-7b1baa912305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97793a227d354a99a9f82d99f3ffd93d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:32:11.946387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:32:11.956487 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 740/1507] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:32:11 2022] DELETE /identity/v3/users/97793a227d354a99a9f82d99f3ffd93d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:32:11.961816 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356925c-1e25-4e9f-9f4c-670a345ed171 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:32:11.962394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356925c-1e25-4e9f-9f4c-670a345ed171 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:32:11.962853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356925c-1e25-4e9f-9f4c-670a345ed171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:32:11.982802 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-9356925c-1e25-4e9f-9f4c-670a345ed171 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:32:11.984687 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-9356925c-1e25-4e9f-9f4c-670a345ed171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:32:12.043181 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 768/1508] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:32:12.054076 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d409e0-83b8-450e-bfd3-7b1baa912305 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:32:12.055559 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-50d409e0-83b8-450e-bfd3-7b1baa912305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:32:12.112848 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d409e0-83b8-450e-bfd3-7b1baa912305 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:32:12.116319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28b770-5bc3-4ae8-96f9-825010dcb3b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:32:12.117006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28b770-5bc3-4ae8-96f9-825010dcb3b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:32:12.117312 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28b770-5bc3-4ae8-96f9-825010dcb3b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:32:12.117590 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28b770-5bc3-4ae8-96f9-825010dcb3b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:32:12.128526 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28b770-5bc3-4ae8-96f9-825010dcb3b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:32:12.130138 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28b770-5bc3-4ae8-96f9-825010dcb3b8 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:32:12.140832 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 741/1509] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:32:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:32:12.497345 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356925c-1e25-4e9f-9f4c-670a345ed171 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:32:12.498876 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-9356925c-1e25-4e9f-9f4c-670a345ed171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:32:12.511984 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356925c-1e25-4e9f-9f4c-670a345ed171 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:32:12.518664 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fc5cf-0d55-4812-9bd7-82497733ea59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:32:12.519564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fc5cf-0d55-4812-9bd7-82497733ea59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:32:12.519845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fc5cf-0d55-4812-9bd7-82497733ea59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:32:12.520110 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fc5cf-0d55-4812-9bd7-82497733ea59 admin admin] PATH_INFO: `/v3/projects/fb06b0948d804de086aca1cfddea1454` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:32:12.522438 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fc5cf-0d55-4812-9bd7-82497733ea59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb06b0948d804de086aca1cfddea1454)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:32:12.523491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: WARNING py.warnings [None req-e04fc5cf-0d55-4812-9bd7-82497733ea59 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:32:12.523491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: warnings.warn(msg) Jun 23 22:32:12.523491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: Jun 23 22:32:12.524700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fc5cf-0d55-4812-9bd7-82497733ea59 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:32:12.587712 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e04fc5cf-0d55-4812-9bd7-82497733ea59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb06b0948d804de086aca1cfddea1454', 'request_id': 'req-e04fc5cf-0d55-4812-9bd7-82497733ea59'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:32:12.601370 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e04fc5cf-0d55-4812-9bd7-82497733ea59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb06b0948d804de086aca1cfddea1454', 'request_id': 'req-e04fc5cf-0d55-4812-9bd7-82497733ea59'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:32:12.611907 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e04fc5cf-0d55-4812-9bd7-82497733ea59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb06b0948d804de086aca1cfddea1454', 'request_id': 'req-e04fc5cf-0d55-4812-9bd7-82497733ea59'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:32:12.623178 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e04fc5cf-0d55-4812-9bd7-82497733ea59 admin admin] The token cache is being invalidate because project fb06b0948d804de086aca1cfddea1454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:32:12.623453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e04fc5cf-0d55-4812-9bd7-82497733ea59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:32:12.624150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e04fc5cf-0d55-4812-9bd7-82497733ea59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:32:12.625909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 769/1510] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:32:12 2022] DELETE /identity/v3/projects/fb06b0948d804de086aca1cfddea1454 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:32:22.136048 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:32:22.619918 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:33:52.929179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd2504-51d0-457a-a908-b6a097bcfd91 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:33:52.929767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd2504-51d0-457a-a908-b6a097bcfd91 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:33:52.929767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd2504-51d0-457a-a908-b6a097bcfd91 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:33:52.931215 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 742/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:33:52.936163 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864ee5a-6279-4b5b-9bd7-aade3f5fd367 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:33:52.936616 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864ee5a-6279-4b5b-9bd7-aade3f5fd367 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:33:52.936718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864ee5a-6279-4b5b-9bd7-aade3f5fd367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:33:52.955401 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-4864ee5a-6279-4b5b-9bd7-aade3f5fd367 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:33:52.956603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-4864ee5a-6279-4b5b-9bd7-aade3f5fd367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:33:53.019684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 770/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:33:53.180478 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd2504-51d0-457a-a908-b6a097bcfd91 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:33:53.182263 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-30dd2504-51d0-457a-a908-b6a097bcfd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:33:53.255969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd2504-51d0-457a-a908-b6a097bcfd91 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:33:53.261281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766105a5-9c42-4c66-86a6-2291dcec4902 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:33:53.261933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766105a5-9c42-4c66-86a6-2291dcec4902 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:33:53.262224 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766105a5-9c42-4c66-86a6-2291dcec4902 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:33:53.262508 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766105a5-9c42-4c66-86a6-2291dcec4902 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:33:53.271372 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766105a5-9c42-4c66-86a6-2291dcec4902 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:33:53.272949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766105a5-9c42-4c66-86a6-2291dcec4902 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:33:53.282760 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 743/1513] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:33:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:33:54.129444 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144feeb-c57f-47de-8764-e63bd0c56a7b None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:33:54.129916 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144feeb-c57f-47de-8764-e63bd0c56a7b None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:33:54.130056 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144feeb-c57f-47de-8764-e63bd0c56a7b None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:33:54.131472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 771/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:33:54.136316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951e6e7-784c-4b88-b56d-150e01add3cd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:33:54.136779 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951e6e7-784c-4b88-b56d-150e01add3cd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:33:54.136879 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951e6e7-784c-4b88-b56d-150e01add3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:33:54.157158 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1951e6e7-784c-4b88-b56d-150e01add3cd None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:33:54.158464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1951e6e7-784c-4b88-b56d-150e01add3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:33:54.166379 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 744/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:33:54.320237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144feeb-c57f-47de-8764-e63bd0c56a7b None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:33:54.334569 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144feeb-c57f-47de-8764-e63bd0c56a7b None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:33:54.340227 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a423c-afee-45f9-9bff-cb3dbf39d524 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:33:54.341395 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a423c-afee-45f9-9bff-cb3dbf39d524 service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:33:54.341395 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a423c-afee-45f9-9bff-cb3dbf39d524 service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:33:54.341395 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a423c-afee-45f9-9bff-cb3dbf39d524 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:33:54.350013 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a423c-afee-45f9-9bff-cb3dbf39d524 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:33:54.351654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a423c-afee-45f9-9bff-cb3dbf39d524 service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:33:54.361907 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 772/1516] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:33:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:33:55.724186 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0a542-077f-4c97-9790-e05d41b9b56c None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:33:55.724747 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0a542-077f-4c97-9790-e05d41b9b56c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:33:55.724747 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0a542-077f-4c97-9790-e05d41b9b56c None None] PATH_INFO: `/v3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:33:55.726077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 745/1517] 10.222.0.55 () {54 vars in 925 bytes} [Thu Jun 23 22:33:55 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:33:55.732208 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231508a-3cc6-465c-b02e-d3b555ce680d None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:33:55.732640 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231508a-3cc6-465c-b02e-d3b555ce680d None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:33:55.732783 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231508a-3cc6-465c-b02e-d3b555ce680d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:33:55.750530 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-7231508a-3cc6-465c-b02e-d3b555ce680d None None] MFA Rules not processed for user `101864afd59543e699189917391e6ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:33:55.751771 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7231508a-3cc6-465c-b02e-d3b555ce680d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:33:55.816405 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 773/1518] 10.222.0.55 () {58 vars in 1015 bytes} [Thu Jun 23 22:33:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:33:59.804110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151a114-3239-4b7d-8669-4466fb722527 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:33:59.804674 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151a114-3239-4b7d-8669-4466fb722527 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:33:59.804674 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151a114-3239-4b7d-8669-4466fb722527 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:33:59.806071 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 746/1519] 10.222.0.43 () {54 vars in 960 bytes} [Thu Jun 23 22:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:33:59.811757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993a010-5d35-43f1-8ef5-572e40eda7ca None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:33:59.812148 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993a010-5d35-43f1-8ef5-572e40eda7ca None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:33:59.812148 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e993a010-5d35-43f1-8ef5-572e40eda7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:33:59.830273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-e993a010-5d35-43f1-8ef5-572e40eda7ca None None] MFA Rules not processed for user `101864afd59543e699189917391e6ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:33:59.831498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e993a010-5d35-43f1-8ef5-572e40eda7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:33:59.840011 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 774/1520] 10.222.0.43 () {58 vars in 1057 bytes} [Thu Jun 23 22:33:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:33:59.847497 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151a114-3239-4b7d-8669-4466fb722527 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:33:59.860944 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151a114-3239-4b7d-8669-4466fb722527 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:33:59.865880 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0178e-90ae-41a1-9197-495c6e4bddd8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101864afd59543e699189917391e6ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:33:59.866519 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0178e-90ae-41a1-9197-495c6e4bddd8 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:33:59.866805 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0178e-90ae-41a1-9197-495c6e4bddd8 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:33:59.867115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0178e-90ae-41a1-9197-495c6e4bddd8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:33:59.877264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd0178e-90ae-41a1-9197-495c6e4bddd8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:33:59.878836 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd0178e-90ae-41a1-9197-495c6e4bddd8 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:33:59.888359 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 747/1521] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:33:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:34:05.611904 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d01fb-7072-48ff-b5d6-e2d47b1ed7f1 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:34:05.612816 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d01fb-7072-48ff-b5d6-e2d47b1ed7f1 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:34:05.613112 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d01fb-7072-48ff-b5d6-e2d47b1ed7f1 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:34:05.614483 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 775/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:34:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:34:05.619359 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ed378-142c-40c6-8477-acb896ffac1c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:34:05.619768 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ed378-142c-40c6-8477-acb896ffac1c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:34:05.619925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ed378-142c-40c6-8477-acb896ffac1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:34:05.640146 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1d6ed378-142c-40c6-8477-acb896ffac1c None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:34:05.641371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1d6ed378-142c-40c6-8477-acb896ffac1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:34:05.649289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 748/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:34:05.808051 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d01fb-7072-48ff-b5d6-e2d47b1ed7f1 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:34:05.822775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d01fb-7072-48ff-b5d6-e2d47b1ed7f1 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:34:05.828920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7b4ab-8b91-428e-81ec-b963c1a6369d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:34:05.829563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7b4ab-8b91-428e-81ec-b963c1a6369d service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:34:05.829854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7b4ab-8b91-428e-81ec-b963c1a6369d service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:34:05.830139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7b4ab-8b91-428e-81ec-b963c1a6369d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:34:05.838218 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a7b4ab-8b91-428e-81ec-b963c1a6369d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:34:05.839846 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a7b4ab-8b91-428e-81ec-b963c1a6369d service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:34:05.849163 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 776/1524] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:34:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:34:06.678140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c132766-5551-43cb-8bd0-cc9b37dc7fab None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:34:06.678728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c132766-5551-43cb-8bd0-cc9b37dc7fab None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:34:06.678728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c132766-5551-43cb-8bd0-cc9b37dc7fab None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:34:06.680742 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 749/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 22:34:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:34:06.685653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99c53c-3f36-44a6-9bf0-68d6c7b6564b None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:34:06.686119 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99c53c-3f36-44a6-9bf0-68d6c7b6564b None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:34:06.686334 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99c53c-3f36-44a6-9bf0-68d6c7b6564b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:34:06.703652 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-2d99c53c-3f36-44a6-9bf0-68d6c7b6564b None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:34:06.704848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2d99c53c-3f36-44a6-9bf0-68d6c7b6564b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:34:06.712442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 777/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 22:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:34:06.865353 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c132766-5551-43cb-8bd0-cc9b37dc7fab None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:34:06.879937 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c132766-5551-43cb-8bd0-cc9b37dc7fab None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:34:06.885964 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb3215-4645-4fb4-8949-b5fd6125ad9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:34:06.886960 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb3215-4645-4fb4-8949-b5fd6125ad9d service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:34:06.887390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb3215-4645-4fb4-8949-b5fd6125ad9d service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:34:06.887782 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb3215-4645-4fb4-8949-b5fd6125ad9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:34:06.896956 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb3215-4645-4fb4-8949-b5fd6125ad9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:34:06.898553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb3215-4645-4fb4-8949-b5fd6125ad9d service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:34:06.908312 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 750/1527] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:34:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:34:10.934895 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa5d3e-a8dc-49f4-b9fb-f92a91ef21cd None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:34:10.936123 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa5d3e-a8dc-49f4-b9fb-f92a91ef21cd None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:34:10.936123 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa5d3e-a8dc-49f4-b9fb-f92a91ef21cd None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:34:10.937510 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 778/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:34:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:34:10.943274 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef12395-9cba-4b66-8fc4-4aedb7ebb9de None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:34:10.943725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef12395-9cba-4b66-8fc4-4aedb7ebb9de None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:34:10.943900 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef12395-9cba-4b66-8fc4-4aedb7ebb9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:34:10.961705 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1ef12395-9cba-4b66-8fc4-4aedb7ebb9de None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:34:10.962901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1ef12395-9cba-4b66-8fc4-4aedb7ebb9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:34:10.970751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 751/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:34:11.008073 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa5d3e-a8dc-49f4-b9fb-f92a91ef21cd None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:34:11.009762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d3aa5d3e-a8dc-49f4-b9fb-f92a91ef21cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:34:11.066194 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa5d3e-a8dc-49f4-b9fb-f92a91ef21cd None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:34:11.069607 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9ed82-db39-41f2-9fca-e4cca2ded32e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:34:11.070248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9ed82-db39-41f2-9fca-e4cca2ded32e service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:34:11.070532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9ed82-db39-41f2-9fca-e4cca2ded32e service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:34:11.070814 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9ed82-db39-41f2-9fca-e4cca2ded32e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:34:11.080260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9ed82-db39-41f2-9fca-e4cca2ded32e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:34:11.081683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9ed82-db39-41f2-9fca-e4cca2ded32e service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:34:11.086485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:34:11.091630 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 779/1530] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:34:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:34:13.140066 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd353153-dcee-4d2e-afb3-5e6610b21a43 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:34:13.140495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd353153-dcee-4d2e-afb3-5e6610b21a43 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:34:13.140621 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd353153-dcee-4d2e-afb3-5e6610b21a43 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:34:13.142044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 752/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:34:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:34:13.147067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb652a-2f87-444e-acf2-b79fb5259330 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:34:13.147494 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb652a-2f87-444e-acf2-b79fb5259330 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:34:13.147629 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb652a-2f87-444e-acf2-b79fb5259330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:34:13.166088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-b1eb652a-2f87-444e-acf2-b79fb5259330 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:34:13.167310 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b1eb652a-2f87-444e-acf2-b79fb5259330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:34:13.175540 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 780/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:34:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:34:13.210571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd353153-dcee-4d2e-afb3-5e6610b21a43 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:34:13.221773 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd353153-dcee-4d2e-afb3-5e6610b21a43 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:34:13.225537 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4e4db-bb3f-4fb7-afe9-243e6d2e78d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:34:13.226184 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4e4db-bb3f-4fb7-afe9-243e6d2e78d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:34:13.226473 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4e4db-bb3f-4fb7-afe9-243e6d2e78d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:34:13.226757 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4e4db-bb3f-4fb7-afe9-243e6d2e78d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:34:13.236399 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4e4db-bb3f-4fb7-afe9-243e6d2e78d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:34:13.237809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4e4db-bb3f-4fb7-afe9-243e6d2e78d3 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:34:13.247445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 753/1533] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:34:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:34:16.799684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d1a69-ea7f-4cb6-b083-03a547a60ece None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:34:16.800269 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d1a69-ea7f-4cb6-b083-03a547a60ece None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:34:16.800269 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d1a69-ea7f-4cb6-b083-03a547a60ece None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:34:16.801608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 781/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:34:16.806824 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d051cc-0878-495c-81bc-96a566d5895e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:34:16.807260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d051cc-0878-495c-81bc-96a566d5895e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:34:16.807436 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d051cc-0878-495c-81bc-96a566d5895e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:34:16.827174 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-38d051cc-0878-495c-81bc-96a566d5895e None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:34:16.828415 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-38d051cc-0878-495c-81bc-96a566d5895e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:34:16.836142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 754/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:34:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:34:17.139698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d1a69-ea7f-4cb6-b083-03a547a60ece None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:34:17.141359 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-929d1a69-ea7f-4cb6-b083-03a547a60ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:34:17.198409 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d1a69-ea7f-4cb6-b083-03a547a60ece None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:34:17.203017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e70c8-c175-4aa3-afc5-6013a832de05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:34:17.203668 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e70c8-c175-4aa3-afc5-6013a832de05 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:34:17.203957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e70c8-c175-4aa3-afc5-6013a832de05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:34:17.204239 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e70c8-c175-4aa3-afc5-6013a832de05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:34:17.212959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529e70c8-c175-4aa3-afc5-6013a832de05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:34:17.214382 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529e70c8-c175-4aa3-afc5-6013a832de05 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:34:17.224311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 782/1536] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:34:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:34:19.255847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722f6d6-9b16-4fc7-b298-a8e0eb6f449d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:34:19.256281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722f6d6-9b16-4fc7-b298-a8e0eb6f449d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:34:19.256449 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722f6d6-9b16-4fc7-b298-a8e0eb6f449d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:34:19.257838 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 755/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 22:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:34:19.262860 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03d178-ffbf-42a2-92f4-614d49165ed4 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:34:19.263301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03d178-ffbf-42a2-92f4-614d49165ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:34:19.263530 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03d178-ffbf-42a2-92f4-614d49165ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:34:19.283805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-dd03d178-ffbf-42a2-92f4-614d49165ed4 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:34:19.285014 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-dd03d178-ffbf-42a2-92f4-614d49165ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:34:19.292648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 783/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 22:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:34:19.590874 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722f6d6-9b16-4fc7-b298-a8e0eb6f449d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:34:19.592898 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8722f6d6-9b16-4fc7-b298-a8e0eb6f449d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:34:19.605820 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722f6d6-9b16-4fc7-b298-a8e0eb6f449d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:34:19.610442 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fc3c5-1b29-4d8d-a6d3-4b99f9b4d919 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:34:19.611109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fc3c5-1b29-4d8d-a6d3-4b99f9b4d919 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:34:19.611401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fc3c5-1b29-4d8d-a6d3-4b99f9b4d919 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:34:19.611678 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fc3c5-1b29-4d8d-a6d3-4b99f9b4d919 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:34:19.620475 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0fc3c5-1b29-4d8d-a6d3-4b99f9b4d919 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:34:19.621883 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0fc3c5-1b29-4d8d-a6d3-4b99f9b4d919 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:34:19.632331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 756/1539] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:34:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:34:29.276586 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=32 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:34:29.628223 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=70 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:48.184562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a555185-6112-49da-bdef-c1c6b2397a60 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.185327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a555185-6112-49da-bdef-c1c6b2397a60 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.185327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a555185-6112-49da-bdef-c1c6b2397a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.208741 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-6a555185-6112-49da-bdef-c1c6b2397a60 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:48.210329 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-6a555185-6112-49da-bdef-c1c6b2397a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:48.228320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 784/1540] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:48.238216 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fcf18-6d70-43cf-b04c-95bb5542a7db None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.239437 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fcf18-6d70-43cf-b04c-95bb5542a7db None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.239926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756fcf18-6d70-43cf-b04c-95bb5542a7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.264967 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-756fcf18-6d70-43cf-b04c-95bb5542a7db None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:48.266812 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-756fcf18-6d70-43cf-b04c-95bb5542a7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:48.277654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 757/1541] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:48.283519 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a555185-6112-49da-bdef-c1c6b2397a60 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:48.295413 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a555185-6112-49da-bdef-c1c6b2397a60 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:48.304355 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10e446-4440-4d7e-a03b-47a900bbac87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:48.305422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10e446-4440-4d7e-a03b-47a900bbac87 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.305902 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10e446-4440-4d7e-a03b-47a900bbac87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.306544 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10e446-4440-4d7e-a03b-47a900bbac87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.307545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10e446-4440-4d7e-a03b-47a900bbac87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:35:48.308062 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10e446-4440-4d7e-a03b-47a900bbac87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:48.310314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10e446-4440-4d7e-a03b-47a900bbac87 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:48.324645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 785/1542] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:35:48.334007 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756fcf18-6d70-43cf-b04c-95bb5542a7db None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:48.347813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756fcf18-6d70-43cf-b04c-95bb5542a7db None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:48.359962 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5ec5e-97fe-49d3-853c-cc57b2d0e22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:48.360929 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5ec5e-97fe-49d3-853c-cc57b2d0e22b admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.361408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5ec5e-97fe-49d3-853c-cc57b2d0e22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.361889 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5ec5e-97fe-49d3-853c-cc57b2d0e22b admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.362778 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5ec5e-97fe-49d3-853c-cc57b2d0e22b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:48.365212 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5ec5e-97fe-49d3-853c-cc57b2d0e22b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:48.420104 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 758/1543] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:35:48 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:48.424087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10e446-4440-4d7e-a03b-47a900bbac87 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:48.434980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10e446-4440-4d7e-a03b-47a900bbac87 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:48.499104 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce97034-fbc3-46c5-a28f-30ef623965c2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.499754 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce97034-fbc3-46c5-a28f-30ef623965c2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.500239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce97034-fbc3-46c5-a28f-30ef623965c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.548063 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5ce97034-fbc3-46c5-a28f-30ef623965c2 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:48.549339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5ce97034-fbc3-46c5-a28f-30ef623965c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:48.556872 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8c690-da39-49bf-970f-ffd3d5d1bafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:48.557577 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8c690-da39-49bf-970f-ffd3d5d1bafe admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.557873 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8c690-da39-49bf-970f-ffd3d5d1bafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.558159 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8c690-da39-49bf-970f-ffd3d5d1bafe admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.562553 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8c690-da39-49bf-970f-ffd3d5d1bafe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:48.565475 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 759/1544] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:48.566569 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cfb2d-aaa0-4dbc-acad-6672032b30b3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.566869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cfb2d-aaa0-4dbc-acad-6672032b30b3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.567185 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45cfb2d-aaa0-4dbc-acad-6672032b30b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.569686 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8c690-da39-49bf-970f-ffd3d5d1bafe admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:48.606709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 786/1545] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:35:48 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:35:48.608610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938e1ad-834e-411b-a098-82a2cb7f2f66 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.609169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938e1ad-834e-411b-a098-82a2cb7f2f66 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.609666 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938e1ad-834e-411b-a098-82a2cb7f2f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.610705 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c45cfb2d-aaa0-4dbc-acad-6672032b30b3 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:48.613110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c45cfb2d-aaa0-4dbc-acad-6672032b30b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:48.640647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 760/1546] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:48.642209 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f396745-2755-4594-b23f-7be29e1a81ef None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.642508 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f396745-2755-4594-b23f-7be29e1a81ef None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.642793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f396745-2755-4594-b23f-7be29e1a81ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.650672 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-b938e1ad-834e-411b-a098-82a2cb7f2f66 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:48.652227 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b938e1ad-834e-411b-a098-82a2cb7f2f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:48.667707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 787/1547] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:48.669586 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938e1ad-834e-411b-a098-82a2cb7f2f66 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:48.681770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938e1ad-834e-411b-a098-82a2cb7f2f66 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:48.685655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5f396745-2755-4594-b23f-7be29e1a81ef None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:48.685655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5f396745-2755-4594-b23f-7be29e1a81ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:48.695724 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccd5b6-a2e0-4d94-a7f6-2f719cf3f642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:48.697074 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccd5b6-a2e0-4d94-a7f6-2f719cf3f642 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.697592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccd5b6-a2e0-4d94-a7f6-2f719cf3f642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.698332 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccd5b6-a2e0-4d94-a7f6-2f719cf3f642 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.699225 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ccd5b6-a2e0-4d94-a7f6-2f719cf3f642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:48.701677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ccd5b6-a2e0-4d94-a7f6-2f719cf3f642 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:48.706350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 761/1548] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:48.708021 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a792a8-2481-499c-be1a-8274a40b5260 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.708316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a792a8-2481-499c-be1a-8274a40b5260 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.708603 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a792a8-2481-499c-be1a-8274a40b5260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.721920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 788/1549] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:48.723274 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccd5b6-a2e0-4d94-a7f6-2f719cf3f642 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:48.739276 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccd5b6-a2e0-4d94-a7f6-2f719cf3f642 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:48.744133 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c2a792a8-2481-499c-be1a-8274a40b5260 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:48.745490 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c2a792a8-2481-499c-be1a-8274a40b5260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:48.749428 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6c6f0-f0d3-4013-93e8-f178dd64017d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:48.750294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6c6f0-f0d3-4013-93e8-f178dd64017d admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.750773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6c6f0-f0d3-4013-93e8-f178dd64017d admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.751245 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6c6f0-f0d3-4013-93e8-f178dd64017d admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.752151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6c6f0-f0d3-4013-93e8-f178dd64017d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:35:48.752470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6c6f0-f0d3-4013-93e8-f178dd64017d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:48.754092 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6c6f0-f0d3-4013-93e8-f178dd64017d admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:48.757010 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 762/1550] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:48.758550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f1854-80a6-420d-8989-f61ccdd71e7a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.758852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f1854-80a6-420d-8989-f61ccdd71e7a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.759170 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f1854-80a6-420d-8989-f61ccdd71e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.774685 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 789/1551] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:35:48.774685 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b9c68-1361-4156-9e3c-99972ebe4e5a None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.774685 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b9c68-1361-4156-9e3c-99972ebe4e5a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.774685 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b9c68-1361-4156-9e3c-99972ebe4e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.779990 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-865f1854-80a6-420d-8989-f61ccdd71e7a None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:48.781268 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-865f1854-80a6-420d-8989-f61ccdd71e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:48.787487 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-456b9c68-1361-4156-9e3c-99972ebe4e5a None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:48.788651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-456b9c68-1361-4156-9e3c-99972ebe4e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:48.789668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 763/1552] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:48.790555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f1854-80a6-420d-8989-f61ccdd71e7a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:48.801778 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 790/1553] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:48.803173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c98aa-e15f-4c38-9efb-24e883ab00be None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.803470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c98aa-e15f-4c38-9efb-24e883ab00be None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.803758 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c98aa-e15f-4c38-9efb-24e883ab00be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.804981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f1854-80a6-420d-8989-f61ccdd71e7a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:48.809851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5135cd9-3d03-47d8-bc39-2cfe815ef4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:48.811382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5135cd9-3d03-47d8-bc39-2cfe815ef4ea admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.812583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5135cd9-3d03-47d8-bc39-2cfe815ef4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.812880 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5135cd9-3d03-47d8-bc39-2cfe815ef4ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.813625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5135cd9-3d03-47d8-bc39-2cfe815ef4ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:48.815274 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5135cd9-3d03-47d8-bc39-2cfe815ef4ea admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:48.822307 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-1b7c98aa-e15f-4c38-9efb-24e883ab00be None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:48.823548 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1b7c98aa-e15f-4c38-9efb-24e883ab00be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:48.827007 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 764/1554] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:48.829200 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a9eaa-c104-4071-ba93-bd2b46f3792c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.829593 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a9eaa-c104-4071-ba93-bd2b46f3792c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.829886 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a9eaa-c104-4071-ba93-bd2b46f3792c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.835632 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 791/1555] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:48.837909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bea0e7-94db-4d36-8f5d-9d6550558f1b None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.838207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bea0e7-94db-4d36-8f5d-9d6550558f1b None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.839372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bea0e7-94db-4d36-8f5d-9d6550558f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.849701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c77a9eaa-c104-4071-ba93-bd2b46f3792c None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:48.850978 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c77a9eaa-c104-4071-ba93-bd2b46f3792c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:48.856924 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-29bea0e7-94db-4d36-8f5d-9d6550558f1b None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:48.858853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-29bea0e7-94db-4d36-8f5d-9d6550558f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:48.859807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 765/1556] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:48.860626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a9eaa-c104-4071-ba93-bd2b46f3792c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:48.870129 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a9eaa-c104-4071-ba93-bd2b46f3792c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:48.870505 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 792/1557] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:48.875741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194514c-6bc7-4e49-bb5c-d60a70d94908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:48.875911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bea0e7-94db-4d36-8f5d-9d6550558f1b None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:48.876413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194514c-6bc7-4e49-bb5c-d60a70d94908 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.876707 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194514c-6bc7-4e49-bb5c-d60a70d94908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.876993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194514c-6bc7-4e49-bb5c-d60a70d94908 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.877726 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194514c-6bc7-4e49-bb5c-d60a70d94908 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:35:48.878074 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194514c-6bc7-4e49-bb5c-d60a70d94908 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:48.879128 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-e194514c-6bc7-4e49-bb5c-d60a70d94908 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:35:48.879128 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Jun 23 22:35:48.879128 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: Jun 23 22:35:48.880014 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194514c-6bc7-4e49-bb5c-d60a70d94908 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:48.885778 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bea0e7-94db-4d36-8f5d-9d6550558f1b None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:48.891121 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a885bb-374a-430d-a213-f55e59cf4b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:48.891781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a885bb-374a-430d-a213-f55e59cf4b87 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.892071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a885bb-374a-430d-a213-f55e59cf4b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.892589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 766/1558] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:35:48.892716 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a885bb-374a-430d-a213-f55e59cf4b87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.893259 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a885bb-374a-430d-a213-f55e59cf4b87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:48.894291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: WARNING py.warnings [None req-a7a885bb-374a-430d-a213-f55e59cf4b87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:35:48.894291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: warnings.warn(msg) Jun 23 22:35:48.894291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: Jun 23 22:35:48.895514 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a885bb-374a-430d-a213-f55e59cf4b87 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:48.901290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfff342-8632-4a94-9f3e-c4bc66139bd8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.901642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfff342-8632-4a94-9f3e-c4bc66139bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.902107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfff342-8632-4a94-9f3e-c4bc66139bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.924465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 793/1559] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:35:48 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:48.925735 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8dfff342-8632-4a94-9f3e-c4bc66139bd8 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:48.925962 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11412d2b-0835-4770-a309-28bb8c94c7bf None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:48.926209 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11412d2b-0835-4770-a309-28bb8c94c7bf None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:48.926498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11412d2b-0835-4770-a309-28bb8c94c7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:48.926940 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8dfff342-8632-4a94-9f3e-c4bc66139bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:48.944146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-11412d2b-0835-4770-a309-28bb8c94c7bf None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:48.945041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-11412d2b-0835-4770-a309-28bb8c94c7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:48.988898 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 767/1560] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:48.991803 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfff342-8632-4a94-9f3e-c4bc66139bd8 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:48.997818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:49.002764 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfff342-8632-4a94-9f3e-c4bc66139bd8 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.007566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16fc7d-37f7-496c-843f-55b1fa078d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.007721 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 794/1561] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:49.008501 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16fc7d-37f7-496c-843f-55b1fa078d6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.008628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11412d2b-0835-4770-a309-28bb8c94c7bf None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.008799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16fc7d-37f7-496c-843f-55b1fa078d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.009026 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16fc7d-37f7-496c-843f-55b1fa078d6a admin admin] PATH_INFO: `/v3/projects/c751cc6b8ac64f3fb7adc9bc71575bd2/users/f53284773f7a430db3d0e16d3c2cd262/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.020825 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11412d2b-0835-4770-a309-28bb8c94c7bf None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.025426 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae2caa-07af-4f82-8da8-48c4bf07ab67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.026092 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae2caa-07af-4f82-8da8-48c4bf07ab67 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.026385 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae2caa-07af-4f82-8da8-48c4bf07ab67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.026693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae2caa-07af-4f82-8da8-48c4bf07ab67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.026953 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16fc7d-37f7-496c-843f-55b1fa078d6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c751cc6b8ac64f3fb7adc9bc71575bd2, user_id=f53284773f7a430db3d0e16d3c2cd262, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.027455 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae2caa-07af-4f82-8da8-48c4bf07ab67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:35:49.027771 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae2caa-07af-4f82-8da8-48c4bf07ab67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.030341 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16fc7d-37f7-496c-843f-55b1fa078d6a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.030457 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae2caa-07af-4f82-8da8-48c4bf07ab67 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.042876 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 795/1562] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:35:49.044471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54b381-2245-4831-a421-33b73e71b4d4 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.044790 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54b381-2245-4831-a421-33b73e71b4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.045060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54b381-2245-4831-a421-33b73e71b4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.049588 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 768/1563] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:35:48 2022] PUT /identity/v3/projects/c751cc6b8ac64f3fb7adc9bc71575bd2/users/f53284773f7a430db3d0e16d3c2cd262/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:35:49.051419 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa0283-62b5-4515-a95f-ee139e78f902 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.051955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa0283-62b5-4515-a95f-ee139e78f902 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.052426 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa0283-62b5-4515-a95f-ee139e78f902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.064684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-1d54b381-2245-4831-a421-33b73e71b4d4 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:49.065972 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1d54b381-2245-4831-a421-33b73e71b4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:49.070110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-deaa0283-62b5-4515-a95f-ee139e78f902 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:49.071322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-deaa0283-62b5-4515-a95f-ee139e78f902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:49.113770 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:49.128512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 796/1564] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:49.129150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54b381-2245-4831-a421-33b73e71b4d4 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.138708 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54b381-2245-4831-a421-33b73e71b4d4 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.140539 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 769/1565] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:49.141249 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa0283-62b5-4515-a95f-ee139e78f902 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.144230 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58888866-6729-4628-9160-e6d8357b8b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.144953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58888866-6729-4628-9160-e6d8357b8b42 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.145241 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58888866-6729-4628-9160-e6d8357b8b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.145529 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58888866-6729-4628-9160-e6d8357b8b42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.146487 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58888866-6729-4628-9160-e6d8357b8b42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.149269 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58888866-6729-4628-9160-e6d8357b8b42 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.150413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa0283-62b5-4515-a95f-ee139e78f902 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.155062 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ac965-9eda-4089-868a-466cf9dc92a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.155726 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ac965-9eda-4089-868a-466cf9dc92a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.156018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ac965-9eda-4089-868a-466cf9dc92a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.156301 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ac965-9eda-4089-868a-466cf9dc92a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.156992 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ac965-9eda-4089-868a-466cf9dc92a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.158275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-5c1ac965-9eda-4089-868a-466cf9dc92a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:35:49.158275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Jun 23 22:35:49.158275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: Jun 23 22:35:49.159495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ac965-9eda-4089-868a-466cf9dc92a0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.178981 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 797/1566] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:35:49 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:49.178981 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58888866-6729-4628-9160-e6d8357b8b42 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.180802 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 770/1567] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:35:49 2022] POST /identity/v3/users => generated 296 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:49.181459 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ac965-9eda-4089-868a-466cf9dc92a0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.191763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ac965-9eda-4089-868a-466cf9dc92a0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.241032 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58888866-6729-4628-9160-e6d8357b8b42 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.245784 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78d8ac-2360-4344-bf98-8b9723d0335d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.246455 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78d8ac-2360-4344-bf98-8b9723d0335d admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.246743 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78d8ac-2360-4344-bf98-8b9723d0335d admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.247060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78d8ac-2360-4344-bf98-8b9723d0335d admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.247815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78d8ac-2360-4344-bf98-8b9723d0335d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:35:49.248132 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78d8ac-2360-4344-bf98-8b9723d0335d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.249563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df97864-a2ea-485b-84dd-66e78dd991ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.250011 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78d8ac-2360-4344-bf98-8b9723d0335d admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.250228 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df97864-a2ea-485b-84dd-66e78dd991ba admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.250564 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df97864-a2ea-485b-84dd-66e78dd991ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.250908 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df97864-a2ea-485b-84dd-66e78dd991ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.251931 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df97864-a2ea-485b-84dd-66e78dd991ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:35:49.252714 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df97864-a2ea-485b-84dd-66e78dd991ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.254291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df97864-a2ea-485b-84dd-66e78dd991ba admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.262951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 798/1568] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:35:49.264169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78d8ac-2360-4344-bf98-8b9723d0335d admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.273031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 771/1569] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:35:49.273031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab7861-b86e-4f1a-98cc-320cf2264c1f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.273031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab7861-b86e-4f1a-98cc-320cf2264c1f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.273031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab7861-b86e-4f1a-98cc-320cf2264c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.275268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78d8ac-2360-4344-bf98-8b9723d0335d admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.280764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0608a-4591-446a-8f7d-4c71ce370f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.281428 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0608a-4591-446a-8f7d-4c71ce370f7f admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.281757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0608a-4591-446a-8f7d-4c71ce370f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.282006 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0608a-4591-446a-8f7d-4c71ce370f7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.282985 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0608a-4591-446a-8f7d-4c71ce370f7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.284875 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0608a-4591-446a-8f7d-4c71ce370f7f admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.286889 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a2ab7861-b86e-4f1a-98cc-320cf2264c1f None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:49.288121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a2ab7861-b86e-4f1a-98cc-320cf2264c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:49.299784 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 772/1570] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:49.299784 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab7861-b86e-4f1a-98cc-320cf2264c1f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.308301 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab7861-b86e-4f1a-98cc-320cf2264c1f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.313643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6dcf6-7aaa-46ae-ba23-4de49359c84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.314412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6dcf6-7aaa-46ae-ba23-4de49359c84d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.314711 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6dcf6-7aaa-46ae-ba23-4de49359c84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.315017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca6dcf6-7aaa-46ae-ba23-4de49359c84d admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.315466 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 799/1571] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:35:49 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:49.315743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6dcf6-7aaa-46ae-ba23-4de49359c84d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:35:49.316053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6dcf6-7aaa-46ae-ba23-4de49359c84d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.316300 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0608a-4591-446a-8f7d-4c71ce370f7f admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.317890 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca6dcf6-7aaa-46ae-ba23-4de49359c84d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.331182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 773/1572] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:35:49.331881 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6dcf6-7aaa-46ae-ba23-4de49359c84d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.343959 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca6dcf6-7aaa-46ae-ba23-4de49359c84d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.361944 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:49.386382 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0608a-4591-446a-8f7d-4c71ce370f7f admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.391350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7d1ed-41da-40ba-9e15-81e894e8c88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.392024 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7d1ed-41da-40ba-9e15-81e894e8c88e admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.392312 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7d1ed-41da-40ba-9e15-81e894e8c88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.392596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7d1ed-41da-40ba-9e15-81e894e8c88e admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.393333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7d1ed-41da-40ba-9e15-81e894e8c88e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:35:49.393642 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7d1ed-41da-40ba-9e15-81e894e8c88e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.395333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7d1ed-41da-40ba-9e15-81e894e8c88e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.409988 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a893866-e7bb-4af6-9730-f87afe4e0723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.410749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 800/1573] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:35:49.411125 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a893866-e7bb-4af6-9730-f87afe4e0723 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.411786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7d1ed-41da-40ba-9e15-81e894e8c88e admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.411910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a893866-e7bb-4af6-9730-f87afe4e0723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.412324 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a893866-e7bb-4af6-9730-f87afe4e0723 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.413176 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a893866-e7bb-4af6-9730-f87afe4e0723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.415274 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a893866-e7bb-4af6-9730-f87afe4e0723 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.422751 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7d1ed-41da-40ba-9e15-81e894e8c88e admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.427557 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d2d7b-cae3-4028-a4f9-1c10604c0a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.428206 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d2d7b-cae3-4028-a4f9-1c10604c0a54 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.428477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d2d7b-cae3-4028-a4f9-1c10604c0a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.428771 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d2d7b-cae3-4028-a4f9-1c10604c0a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.429508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d2d7b-cae3-4028-a4f9-1c10604c0a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.431106 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d2d7b-cae3-4028-a4f9-1c10604c0a54 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.444474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 774/1574] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:35:49 2022] POST /identity/v3/users => generated 284 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:49.444474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a893866-e7bb-4af6-9730-f87afe4e0723 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.446643 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 801/1575] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:49.447672 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d2d7b-cae3-4028-a4f9-1c10604c0a54 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.454515 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a893866-e7bb-4af6-9730-f87afe4e0723 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.458048 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d2d7b-cae3-4028-a4f9-1c10604c0a54 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.459121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514ae27-b354-443f-a0e0-6bc665c5aa2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.459775 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514ae27-b354-443f-a0e0-6bc665c5aa2f admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.460064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514ae27-b354-443f-a0e0-6bc665c5aa2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.460346 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514ae27-b354-443f-a0e0-6bc665c5aa2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.461029 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514ae27-b354-443f-a0e0-6bc665c5aa2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.462851 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80621ca6-229d-404e-9c3e-9aed7aeb372a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.463012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514ae27-b354-443f-a0e0-6bc665c5aa2f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.463527 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80621ca6-229d-404e-9c3e-9aed7aeb372a admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.463842 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80621ca6-229d-404e-9c3e-9aed7aeb372a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.464112 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80621ca6-229d-404e-9c3e-9aed7aeb372a admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.464795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80621ca6-229d-404e-9c3e-9aed7aeb372a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.468067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80621ca6-229d-404e-9c3e-9aed7aeb372a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.494317 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 775/1576] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:35:49 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:49.494317 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514ae27-b354-443f-a0e0-6bc665c5aa2f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.494869 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 802/1577] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:35:49 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:49.498612 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80621ca6-229d-404e-9c3e-9aed7aeb372a admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.502018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:49.549822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514ae27-b354-443f-a0e0-6bc665c5aa2f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.563390 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80621ca6-229d-404e-9c3e-9aed7aeb372a admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.568000 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad47178-d481-495e-8e29-a49f87ecf5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.568677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad47178-d481-495e-8e29-a49f87ecf5ec admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.568970 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad47178-d481-495e-8e29-a49f87ecf5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.569257 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad47178-d481-495e-8e29-a49f87ecf5ec admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.569950 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad47178-d481-495e-8e29-a49f87ecf5ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.571909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad47178-d481-495e-8e29-a49f87ecf5ec admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.594013 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 803/1578] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:35:49 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:49.596121 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad47178-d481-495e-8e29-a49f87ecf5ec admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.599378 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691b728-0f25-4e24-ae2e-8b2ef8f5becc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.600030 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691b728-0f25-4e24-ae2e-8b2ef8f5becc service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.600316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691b728-0f25-4e24-ae2e-8b2ef8f5becc service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.600596 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691b728-0f25-4e24-ae2e-8b2ef8f5becc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.605928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad47178-d481-495e-8e29-a49f87ecf5ec admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.609817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691b728-0f25-4e24-ae2e-8b2ef8f5becc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.610979 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ac2c9-fd16-418f-8b45-31444adecc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.611274 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691b728-0f25-4e24-ae2e-8b2ef8f5becc service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.611637 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ac2c9-fd16-418f-8b45-31444adecc2d admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.611929 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ac2c9-fd16-418f-8b45-31444adecc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.612216 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ac2c9-fd16-418f-8b45-31444adecc2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.613111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ac2c9-fd16-418f-8b45-31444adecc2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.615067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ac2c9-fd16-418f-8b45-31444adecc2d admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.620530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 776/1579] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:35:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:35:49.621278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691b728-0f25-4e24-ae2e-8b2ef8f5becc service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.630415 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691b728-0f25-4e24-ae2e-8b2ef8f5becc service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.635791 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa320189-0be5-470b-8ba0-a9825b5ff0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.636457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa320189-0be5-470b-8ba0-a9825b5ff0cd admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.636826 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa320189-0be5-470b-8ba0-a9825b5ff0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.637177 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa320189-0be5-470b-8ba0-a9825b5ff0cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.637886 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa320189-0be5-470b-8ba0-a9825b5ff0cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.640796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa320189-0be5-470b-8ba0-a9825b5ff0cd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.648883 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 804/1580] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:35:49 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:49.648883 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ac2c9-fd16-418f-8b45-31444adecc2d admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.669688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 777/1581] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:35:49 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:49.674181 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa320189-0be5-470b-8ba0-a9825b5ff0cd admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.719992 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ac2c9-fd16-418f-8b45-31444adecc2d admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.728937 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:49.743938 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa320189-0be5-470b-8ba0-a9825b5ff0cd admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.749586 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8e86c-194d-4083-a708-bcd826bc4bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.750182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8e86c-194d-4083-a708-bcd826bc4bfd admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.750469 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8e86c-194d-4083-a708-bcd826bc4bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.750945 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8e86c-194d-4083-a708-bcd826bc4bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.751695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8e86c-194d-4083-a708-bcd826bc4bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.753307 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8e86c-194d-4083-a708-bcd826bc4bfd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.767072 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 778/1582] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:49.767072 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8e86c-194d-4083-a708-bcd826bc4bfd admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.777402 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8e86c-194d-4083-a708-bcd826bc4bfd admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.783624 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c2793-e7c2-4413-b806-732f3ae7f985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.784563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c2793-e7c2-4413-b806-732f3ae7f985 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.785052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c2793-e7c2-4413-b806-732f3ae7f985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.785552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c2793-e7c2-4413-b806-732f3ae7f985 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.786571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c2793-e7c2-4413-b806-732f3ae7f985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.789008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c2793-e7c2-4413-b806-732f3ae7f985 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.791692 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218440a-ebda-4328-82da-b445fa4457a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.792603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218440a-ebda-4328-82da-b445fa4457a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.793081 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218440a-ebda-4328-82da-b445fa4457a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.793697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218440a-ebda-4328-82da-b445fa4457a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.795475 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218440a-ebda-4328-82da-b445fa4457a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.797812 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218440a-ebda-4328-82da-b445fa4457a9 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.809951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 805/1583] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:49.811845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218440a-ebda-4328-82da-b445fa4457a9 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.817975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 779/1584] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:35:49 2022] POST /identity/v3/users => generated 292 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:49.817975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c2793-e7c2-4413-b806-732f3ae7f985 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.825093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218440a-ebda-4328-82da-b445fa4457a9 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.830947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00810f2b-9932-4311-9da6-b79c647243a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.832103 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00810f2b-9932-4311-9da6-b79c647243a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.832262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00810f2b-9932-4311-9da6-b79c647243a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.832546 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00810f2b-9932-4311-9da6-b79c647243a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.833735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00810f2b-9932-4311-9da6-b79c647243a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.835399 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c2793-e7c2-4413-b806-732f3ae7f985 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.836220 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00810f2b-9932-4311-9da6-b79c647243a8 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.841325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a113524-38e3-4ecd-9f69-dee50834f851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.842224 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a113524-38e3-4ecd-9f69-dee50834f851 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.842679 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a113524-38e3-4ecd-9f69-dee50834f851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.843073 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a113524-38e3-4ecd-9f69-dee50834f851 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.844143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a113524-38e3-4ecd-9f69-dee50834f851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.846030 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a113524-38e3-4ecd-9f69-dee50834f851 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.863558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 806/1585] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:35:49 2022] POST /identity/v3/users => generated 287 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:49.868509 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 780/1586] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:49.868509 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a113524-38e3-4ecd-9f69-dee50834f851 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.868825 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00810f2b-9932-4311-9da6-b79c647243a8 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.880518 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00810f2b-9932-4311-9da6-b79c647243a8 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.881258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a113524-38e3-4ecd-9f69-dee50834f851 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.885776 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bf6e2-2a86-4311-a69f-0787a736b5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.886278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd86de7-34c0-4320-8514-ec618a3c513c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.886465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bf6e2-2a86-4311-a69f-0787a736b5b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.887141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bf6e2-2a86-4311-a69f-0787a736b5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.887255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd86de7-34c0-4320-8514-ec618a3c513c admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.887429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bf6e2-2a86-4311-a69f-0787a736b5b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.887741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd86de7-34c0-4320-8514-ec618a3c513c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.888096 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd86de7-34c0-4320-8514-ec618a3c513c admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.888204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bf6e2-2a86-4311-a69f-0787a736b5b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.889251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd86de7-34c0-4320-8514-ec618a3c513c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.890310 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bf6e2-2a86-4311-a69f-0787a736b5b4 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.891386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd86de7-34c0-4320-8514-ec618a3c513c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.903412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:49.915575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 807/1587] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:35:49 2022] POST /identity/v3/users => generated 288 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:49.916679 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bf6e2-2a86-4311-a69f-0787a736b5b4 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.922939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 781/1588] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:35:49 2022] POST /identity/v3/users => generated 287 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:49.923885 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd86de7-34c0-4320-8514-ec618a3c513c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.928615 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bf6e2-2a86-4311-a69f-0787a736b5b4 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.933673 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437238f-7d27-45db-b030-e860858bcb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.934670 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437238f-7d27-45db-b030-e860858bcb81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.934974 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437238f-7d27-45db-b030-e860858bcb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.935884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437238f-7d27-45db-b030-e860858bcb81 admin admin] PATH_INFO: `/v3/projects/192d1c8d15ce462dbf10fc3b519ab253/users/dc246e144afc46698000f7f53ef6489e/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.936870 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd86de7-34c0-4320-8514-ec618a3c513c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.943167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3994a1-d5d3-4b3d-859a-ae21d2d54892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:49.943858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3994a1-d5d3-4b3d-859a-ae21d2d54892 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:49.944153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3994a1-d5d3-4b3d-859a-ae21d2d54892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:49.944441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3994a1-d5d3-4b3d-859a-ae21d2d54892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:49.945787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3994a1-d5d3-4b3d-859a-ae21d2d54892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.947659 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3994a1-d5d3-4b3d-859a-ae21d2d54892 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.952952 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437238f-7d27-45db-b030-e860858bcb81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=192d1c8d15ce462dbf10fc3b519ab253, user_id=dc246e144afc46698000f7f53ef6489e, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:49.954117 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: WARNING py.warnings [None req-c437238f-7d27-45db-b030-e860858bcb81 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:35:49.954117 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: warnings.warn(msg) Jun 23 22:35:49.954117 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: Jun 23 22:35:49.956074 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437238f-7d27-45db-b030-e860858bcb81 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:49.959950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 782/1589] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:49.960819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3994a1-d5d3-4b3d-859a-ae21d2d54892 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:49.972940 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3994a1-d5d3-4b3d-859a-ae21d2d54892 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:49.976334 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 808/1590] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:35:49 2022] PUT /identity/v3/projects/192d1c8d15ce462dbf10fc3b519ab253/users/dc246e144afc46698000f7f53ef6489e/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:35:49.977538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437238f-7d27-45db-b030-e860858bcb81 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:50.036324 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008daf1-595d-4f4f-8789-4eea95c9a7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:50.037129 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008daf1-595d-4f4f-8789-4eea95c9a7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.037555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008daf1-595d-4f4f-8789-4eea95c9a7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.037843 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008daf1-595d-4f4f-8789-4eea95c9a7b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.038571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008daf1-595d-4f4f-8789-4eea95c9a7b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:50.040232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008daf1-595d-4f4f-8789-4eea95c9a7b3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:50.051184 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 783/1591] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:50.052233 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008daf1-595d-4f4f-8789-4eea95c9a7b3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:50.057089 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437238f-7d27-45db-b030-e860858bcb81 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:50.063161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008daf1-595d-4f4f-8789-4eea95c9a7b3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:50.064871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74594eab-44eb-4b42-9d60-8bbcbac795da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:50.065567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74594eab-44eb-4b42-9d60-8bbcbac795da admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.065862 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74594eab-44eb-4b42-9d60-8bbcbac795da admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.066149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74594eab-44eb-4b42-9d60-8bbcbac795da admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.066904 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74594eab-44eb-4b42-9d60-8bbcbac795da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:50.068570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74594eab-44eb-4b42-9d60-8bbcbac795da admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:50.069093 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4db20c-eae4-4637-8f28-8bd14e2556bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:50.069859 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4db20c-eae4-4637-8f28-8bd14e2556bf admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.070215 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4db20c-eae4-4637-8f28-8bd14e2556bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.070582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4db20c-eae4-4637-8f28-8bd14e2556bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.071426 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4db20c-eae4-4637-8f28-8bd14e2556bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:50.073130 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4db20c-eae4-4637-8f28-8bd14e2556bf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:50.081877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 809/1592] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:50.083990 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74594eab-44eb-4b42-9d60-8bbcbac795da admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:50.085917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 784/1593] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:50.087027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4db20c-eae4-4637-8f28-8bd14e2556bf admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:50.097226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74594eab-44eb-4b42-9d60-8bbcbac795da admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:50.099466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4db20c-eae4-4637-8f28-8bd14e2556bf admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:50.102819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdbbacf-8a15-44da-b113-d8b5212956cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:50.103839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbbacf-8a15-44da-b113-d8b5212956cc admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.104283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbbacf-8a15-44da-b113-d8b5212956cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.104808 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3056a6-18c3-43d9-afdb-8e88f4b589a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:50.105032 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbbacf-8a15-44da-b113-d8b5212956cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.105347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3056a6-18c3-43d9-afdb-8e88f4b589a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.105482 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdbbacf-8a15-44da-b113-d8b5212956cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:50.105961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3056a6-18c3-43d9-afdb-8e88f4b589a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.106251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3056a6-18c3-43d9-afdb-8e88f4b589a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.107109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3056a6-18c3-43d9-afdb-8e88f4b589a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:50.107199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdbbacf-8a15-44da-b113-d8b5212956cc admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:50.109263 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3056a6-18c3-43d9-afdb-8e88f4b589a5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:50.122058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 785/1594] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:50.122229 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3056a6-18c3-43d9-afdb-8e88f4b589a5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:50.138279 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 810/1595] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:50.139629 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3056a6-18c3-43d9-afdb-8e88f4b589a5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:50.140682 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db695d4c-8f7e-41e1-bf05-f77f3a2ccb02 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.142151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db695d4c-8f7e-41e1-bf05-f77f3a2ccb02 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.142664 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db695d4c-8f7e-41e1-bf05-f77f3a2ccb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.145785 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f2d66-0558-40e0-bb4c-d68b6899b3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:50.147152 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f2d66-0558-40e0-bb4c-d68b6899b3ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.147549 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f2d66-0558-40e0-bb4c-d68b6899b3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.147912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f2d66-0558-40e0-bb4c-d68b6899b3ab admin admin] PATH_INFO: `/v3/projects/6466e16d7f0d404db3f28d9e93b89b4a/users/85f1409263c5421eaa55e6354761b0d0/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.164421 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-db695d4c-8f7e-41e1-bf05-f77f3a2ccb02 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:50.166022 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-db695d4c-8f7e-41e1-bf05-f77f3a2ccb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:50.168332 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f2d66-0558-40e0-bb4c-d68b6899b3ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=6466e16d7f0d404db3f28d9e93b89b4a, user_id=85f1409263c5421eaa55e6354761b0d0, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:50.171392 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f2d66-0558-40e0-bb4c-d68b6899b3ab admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:50.178767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 811/1596] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:50.180179 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db695d4c-8f7e-41e1-bf05-f77f3a2ccb02 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:50.193801 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 786/1597] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:35:50 2022] PUT /identity/v3/projects/6466e16d7f0d404db3f28d9e93b89b4a/users/85f1409263c5421eaa55e6354761b0d0/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:35:50.194649 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f2d66-0558-40e0-bb4c-d68b6899b3ab admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:50.195818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db695d4c-8f7e-41e1-bf05-f77f3a2ccb02 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:50.222149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:50.261128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea066186-ea8c-4870-b982-5ea7d9f81c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:50.261813 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea066186-ea8c-4870-b982-5ea7d9f81c80 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.262174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea066186-ea8c-4870-b982-5ea7d9f81c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.262522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea066186-ea8c-4870-b982-5ea7d9f81c80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.263390 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea066186-ea8c-4870-b982-5ea7d9f81c80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:50.265237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea066186-ea8c-4870-b982-5ea7d9f81c80 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:50.268713 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f2d66-0558-40e0-bb4c-d68b6899b3ab admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:50.273292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0477019-c37c-4c65-b7ea-623943f0303f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:50.273960 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0477019-c37c-4c65-b7ea-623943f0303f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.274245 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0477019-c37c-4c65-b7ea-623943f0303f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.274521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0477019-c37c-4c65-b7ea-623943f0303f admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.275302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0477019-c37c-4c65-b7ea-623943f0303f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:50.275725 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 812/1598] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:50.276581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea066186-ea8c-4870-b982-5ea7d9f81c80 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:50.280462 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0477019-c37c-4c65-b7ea-623943f0303f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:50.287202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea066186-ea8c-4870-b982-5ea7d9f81c80 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:50.293948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20ce01-185d-4a8d-9937-8e20c1013f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:50.295354 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 787/1599] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:50.295495 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20ce01-185d-4a8d-9937-8e20c1013f8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.295791 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20ce01-185d-4a8d-9937-8e20c1013f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.296126 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20ce01-185d-4a8d-9937-8e20c1013f8c admin admin] PATH_INFO: `/v3/projects/e3870456e11b4171ab9ed057ebdf2088/users/4809dd1bd3ea4bb4a80de8a09f7eec6e/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.296256 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0477019-c37c-4c65-b7ea-623943f0303f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:50.305985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0477019-c37c-4c65-b7ea-623943f0303f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:50.311040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5a330-435b-4249-8c42-237dea642636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:50.311710 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5a330-435b-4249-8c42-237dea642636 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.311999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5a330-435b-4249-8c42-237dea642636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.312279 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5a330-435b-4249-8c42-237dea642636 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.312885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20ce01-185d-4a8d-9937-8e20c1013f8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3870456e11b4171ab9ed057ebdf2088, user_id=4809dd1bd3ea4bb4a80de8a09f7eec6e, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:50.312999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5a330-435b-4249-8c42-237dea642636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:50.314572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5a330-435b-4249-8c42-237dea642636 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:50.315779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20ce01-185d-4a8d-9937-8e20c1013f8c admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:50.329854 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 788/1600] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:50.330911 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5a330-435b-4249-8c42-237dea642636 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:50.331520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 813/1601] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:35:50 2022] PUT /identity/v3/projects/e3870456e11b4171ab9ed057ebdf2088/users/4809dd1bd3ea4bb4a80de8a09f7eec6e/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:35:50.332201 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20ce01-185d-4a8d-9937-8e20c1013f8c admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:50.342779 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5a330-435b-4249-8c42-237dea642636 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:50.363397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:50.396995 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20ce01-185d-4a8d-9937-8e20c1013f8c admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:50.401311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bd301-42c4-4a01-9313-5fbd0d86d500 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:50.402092 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bd301-42c4-4a01-9313-5fbd0d86d500 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.402390 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bd301-42c4-4a01-9313-5fbd0d86d500 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.402671 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bd301-42c4-4a01-9313-5fbd0d86d500 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.410959 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbf04b-cc34-445a-a69b-b35dc91b2389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:50.410959 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbf04b-cc34-445a-a69b-b35dc91b2389 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.410959 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbf04b-cc34-445a-a69b-b35dc91b2389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.410959 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbf04b-cc34-445a-a69b-b35dc91b2389 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.411572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dbf04b-cc34-445a-a69b-b35dc91b2389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:50.413990 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dbf04b-cc34-445a-a69b-b35dc91b2389 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:50.427317 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 789/1602] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:50.432005 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87890cf-8925-4e63-96b3-ae2039a03352 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.432301 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87890cf-8925-4e63-96b3-ae2039a03352 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.432583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87890cf-8925-4e63-96b3-ae2039a03352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.462994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e87890cf-8925-4e63-96b3-ae2039a03352 None None] MFA Rules not processed for user `85f1409263c5421eaa55e6354761b0d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:50.464596 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e87890cf-8925-4e63-96b3-ae2039a03352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:50.487017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182bd301-42c4-4a01-9313-5fbd0d86d500 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:50.489054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182bd301-42c4-4a01-9313-5fbd0d86d500 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:50.504437 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 814/1603] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:35:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:35:50.504437 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bd301-42c4-4a01-9313-5fbd0d86d500 service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:50.515381 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bd301-42c4-4a01-9313-5fbd0d86d500 service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:50.520495 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e32e5a-0b81-44fd-a92f-ec8ad332a5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:50.521409 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e32e5a-0b81-44fd-a92f-ec8ad332a5e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.521697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e32e5a-0b81-44fd-a92f-ec8ad332a5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.522020 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e32e5a-0b81-44fd-a92f-ec8ad332a5e6 admin admin] PATH_INFO: `/v3/projects/2735e40fe2564a6d8a5cdc7c32b0b869/users/03486c7630314392b85271455d12aba2/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.541015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e32e5a-0b81-44fd-a92f-ec8ad332a5e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2735e40fe2564a6d8a5cdc7c32b0b869, user_id=03486c7630314392b85271455d12aba2, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:50.543870 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e32e5a-0b81-44fd-a92f-ec8ad332a5e6 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:50.555386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 790/1604] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:50.556745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87890cf-8925-4e63-96b3-ae2039a03352 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:50.571963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87890cf-8925-4e63-96b3-ae2039a03352 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:50.572500 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 815/1605] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:35:50 2022] PUT /identity/v3/projects/2735e40fe2564a6d8a5cdc7c32b0b869/users/03486c7630314392b85271455d12aba2/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:35:50.573664 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e32e5a-0b81-44fd-a92f-ec8ad332a5e6 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:50.632734 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:50.657589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c40402-1f96-4fb3-b1e1-dd6f66a72de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:50.658724 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c40402-1f96-4fb3-b1e1-dd6f66a72de6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.659124 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c40402-1f96-4fb3-b1e1-dd6f66a72de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.659545 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c40402-1f96-4fb3-b1e1-dd6f66a72de6 admin admin] PATH_INFO: `/v3/projects/5bdfd8b306f24510a91e99586724a2cb/users/ac0dc12e841345a482590080c487acfe/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.689362 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e32e5a-0b81-44fd-a92f-ec8ad332a5e6 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:50.695179 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fa4a1-c52e-44eb-87dc-6f59bcad7dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:50.697394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fa4a1-c52e-44eb-87dc-6f59bcad7dbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.697668 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fa4a1-c52e-44eb-87dc-6f59bcad7dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.697959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fa4a1-c52e-44eb-87dc-6f59bcad7dbc admin admin] PATH_INFO: `/v3/projects/2d484d9303ed401fbd838f2c6bb9c04d/users/ea3c44869fe74d628b8dd5ecd0c22485/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.703702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c40402-1f96-4fb3-b1e1-dd6f66a72de6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bdfd8b306f24510a91e99586724a2cb, user_id=ac0dc12e841345a482590080c487acfe, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:50.712806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c40402-1f96-4fb3-b1e1-dd6f66a72de6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:50.716549 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6fa4a1-c52e-44eb-87dc-6f59bcad7dbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d484d9303ed401fbd838f2c6bb9c04d, user_id=ea3c44869fe74d628b8dd5ecd0c22485, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:50.719742 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6fa4a1-c52e-44eb-87dc-6f59bcad7dbc admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:50.736645 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 791/1606] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:35:50 2022] PUT /identity/v3/projects/5bdfd8b306f24510a91e99586724a2cb/users/ac0dc12e841345a482590080c487acfe/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:35:50.738108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423beba7-2617-4978-943a-ed08f07959fd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.738416 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423beba7-2617-4978-943a-ed08f07959fd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.738700 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423beba7-2617-4978-943a-ed08f07959fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.749486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 816/1607] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:35:50 2022] PUT /identity/v3/projects/2d484d9303ed401fbd838f2c6bb9c04d/users/ea3c44869fe74d628b8dd5ecd0c22485/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:35:50.749486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fa4a1-c52e-44eb-87dc-6f59bcad7dbc admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:50.800138 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-423beba7-2617-4978-943a-ed08f07959fd None None] MFA Rules not processed for user `4809dd1bd3ea4bb4a80de8a09f7eec6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:50.802074 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-423beba7-2617-4978-943a-ed08f07959fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:50.825764 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:50.835327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fa4a1-c52e-44eb-87dc-6f59bcad7dbc admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:50.843667 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73890039-8dfa-4319-abdc-abbc938791c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:50.856603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73890039-8dfa-4319-abdc-abbc938791c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.858519 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73890039-8dfa-4319-abdc-abbc938791c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.858993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73890039-8dfa-4319-abdc-abbc938791c3 admin admin] PATH_INFO: `/v3/projects/becdae52e5c64b1ea2bb05833505b0f8/users/53c519b28b174ef385dd7e76b85cf504/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.891746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73890039-8dfa-4319-abdc-abbc938791c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=becdae52e5c64b1ea2bb05833505b0f8, user_id=53c519b28b174ef385dd7e76b85cf504, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:50.902004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73890039-8dfa-4319-abdc-abbc938791c3 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:50.923056 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 792/1608] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:50.924192 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423beba7-2617-4978-943a-ed08f07959fd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:50.939793 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 817/1609] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:35:50 2022] PUT /identity/v3/projects/becdae52e5c64b1ea2bb05833505b0f8/users/53c519b28b174ef385dd7e76b85cf504/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:35:50.944256 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb54fa6-92a9-4acb-bc5c-e5adb6949013 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:50.945122 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb54fa6-92a9-4acb-bc5c-e5adb6949013 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:50.945625 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb54fa6-92a9-4acb-bc5c-e5adb6949013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:50.985371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-bcb54fa6-92a9-4acb-bc5c-e5adb6949013 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:50.986664 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-bcb54fa6-92a9-4acb-bc5c-e5adb6949013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:51.037223 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423beba7-2617-4978-943a-ed08f07959fd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:51.082513 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 818/1610] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:51.085383 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3e562-9f80-42b8-a1ab-0ded1d7cf8a9 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:51.085750 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3e562-9f80-42b8-a1ab-0ded1d7cf8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:51.086113 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3e562-9f80-42b8-a1ab-0ded1d7cf8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:51.143662 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8c465-85f7-4747-bc8d-21150bd32442 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:51.144339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8c465-85f7-4747-bc8d-21150bd32442 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:51.144628 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8c465-85f7-4747-bc8d-21150bd32442 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:51.144912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8c465-85f7-4747-bc8d-21150bd32442 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:51.158385 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-55b3e562-9f80-42b8-a1ab-0ded1d7cf8a9 None None] MFA Rules not processed for user `ac0dc12e841345a482590080c487acfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:51.162460 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-55b3e562-9f80-42b8-a1ab-0ded1d7cf8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:51.218116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:51.238302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8c465-85f7-4747-bc8d-21150bd32442 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:51.239849 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8c465-85f7-4747-bc8d-21150bd32442 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:51.253407 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 793/1611] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:35:50 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:35:51.255491 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e996e8-1d05-4d94-81f2-8894e16f5ece None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:51.255798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e996e8-1d05-4d94-81f2-8894e16f5ece None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:51.256421 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e996e8-1d05-4d94-81f2-8894e16f5ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:51.319636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-52e996e8-1d05-4d94-81f2-8894e16f5ece None None] MFA Rules not processed for user `ea3c44869fe74d628b8dd5ecd0c22485`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:51.322396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-52e996e8-1d05-4d94-81f2-8894e16f5ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:51.334308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 819/1612] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:51.336881 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02ff23-c777-4bd2-9b8e-5897e588a7d9 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:51.338304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02ff23-c777-4bd2-9b8e-5897e588a7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:51.338606 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd02ff23-c777-4bd2-9b8e-5897e588a7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:51.369670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:51.380171 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-dd02ff23-c777-4bd2-9b8e-5897e588a7d9 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:51.381397 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-dd02ff23-c777-4bd2-9b8e-5897e588a7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:51.400005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 820/1613] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:51.401353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afe71f-5756-445c-be44-83de8fbefcb4 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:51.401643 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afe71f-5756-445c-be44-83de8fbefcb4 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:51.402088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afe71f-5756-445c-be44-83de8fbefcb4 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:51.419484 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 821/1614] 10.222.0.43 () {54 vars in 961 bytes} [Thu Jun 23 22:35:51 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:35:51.420855 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afe71f-5756-445c-be44-83de8fbefcb4 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:51.490869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 794/1615] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:51.491710 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e996e8-1d05-4d94-81f2-8894e16f5ece None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:51.540462 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e996e8-1d05-4d94-81f2-8894e16f5ece None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:51.555190 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a23c8-3065-411e-9f25-54cb93f0687c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:51.563400 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a23c8-3065-411e-9f25-54cb93f0687c admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:51.563691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a23c8-3065-411e-9f25-54cb93f0687c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:51.563976 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a23c8-3065-411e-9f25-54cb93f0687c admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:51.564694 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a23c8-3065-411e-9f25-54cb93f0687c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:51.566635 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a23c8-3065-411e-9f25-54cb93f0687c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:51.568607 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afe71f-5756-445c-be44-83de8fbefcb4 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:51.576219 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454676bc-9322-48c4-9d11-8cbd6668dd05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:51.577070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454676bc-9322-48c4-9d11-8cbd6668dd05 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:51.577355 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454676bc-9322-48c4-9d11-8cbd6668dd05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:51.577640 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454676bc-9322-48c4-9d11-8cbd6668dd05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:51.610651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454676bc-9322-48c4-9d11-8cbd6668dd05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:51.612250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454676bc-9322-48c4-9d11-8cbd6668dd05 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:51.644144 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 795/1617] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:35:51 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:51.648637 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 822/1617] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:35:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:35:51.650358 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b314c-8fdb-496a-a038-8fa4078cfad1 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:51.650477 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a23c8-3065-411e-9f25-54cb93f0687c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:51.650566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b314c-8fdb-496a-a038-8fa4078cfad1 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:51.651086 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b314c-8fdb-496a-a038-8fa4078cfad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:51.684194 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-274b314c-8fdb-496a-a038-8fa4078cfad1 None None] MFA Rules not processed for user `35d686923ef541819de3c77847d38511`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:51.687305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-274b314c-8fdb-496a-a038-8fa4078cfad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:51.755350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a23c8-3065-411e-9f25-54cb93f0687c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:51.766162 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7538ad6-7cd5-4c6b-8819-0eefa3cc82c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:51.768066 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7538ad6-7cd5-4c6b-8819-0eefa3cc82c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:51.769199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7538ad6-7cd5-4c6b-8819-0eefa3cc82c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:51.769807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7538ad6-7cd5-4c6b-8819-0eefa3cc82c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:51.860544 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 823/1618] 10.222.0.43 () {58 vars in 1058 bytes} [Thu Jun 23 22:35:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:51.874086 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc57f68-93ec-4012-8647-1fcd91552310 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:51.874336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc57f68-93ec-4012-8647-1fcd91552310 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:51.875777 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc57f68-93ec-4012-8647-1fcd91552310 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:51.877236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 824/1619] 10.222.0.43 () {54 vars in 961 bytes} [Thu Jun 23 22:35:51 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:35:51.883729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc57f68-93ec-4012-8647-1fcd91552310 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:51.937561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7538ad6-7cd5-4c6b-8819-0eefa3cc82c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:51.945704 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7538ad6-7cd5-4c6b-8819-0eefa3cc82c3 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:51.969013 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:51.982650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 796/1620] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:35:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:35:51.984086 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7538ad6-7cd5-4c6b-8819-0eefa3cc82c3 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:52.008732 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7538ad6-7cd5-4c6b-8819-0eefa3cc82c3 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:52.014646 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc57f68-93ec-4012-8647-1fcd91552310 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:52.017679 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3019bf3-7ed6-4c26-8f77-6f16d7c8a606 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:52.018583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3019bf3-7ed6-4c26-8f77-6f16d7c8a606 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:52.021325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3019bf3-7ed6-4c26-8f77-6f16d7c8a606 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:52.021325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3019bf3-7ed6-4c26-8f77-6f16d7c8a606 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:52.032487 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78153e92-b499-41ed-af5f-4e606f6691cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:52.036627 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78153e92-b499-41ed-af5f-4e606f6691cc admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:52.036944 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78153e92-b499-41ed-af5f-4e606f6691cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:52.037245 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78153e92-b499-41ed-af5f-4e606f6691cc admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:52.038537 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78153e92-b499-41ed-af5f-4e606f6691cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:52.047362 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78153e92-b499-41ed-af5f-4e606f6691cc admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:52.093647 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 825/1621] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:35:51 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:52.096629 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8b691-9eb3-4480-9aa9-56db14d4bd18 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:52.096939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8b691-9eb3-4480-9aa9-56db14d4bd18 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:52.097234 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8b691-9eb3-4480-9aa9-56db14d4bd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:52.116432 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:52.146731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3019bf3-7ed6-4c26-8f77-6f16d7c8a606 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:52.148718 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3019bf3-7ed6-4c26-8f77-6f16d7c8a606 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:52.165754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-03e8b691-9eb3-4480-9aa9-56db14d4bd18 None None] MFA Rules not processed for user `35d686923ef541819de3c77847d38511`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:52.170071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-03e8b691-9eb3-4480-9aa9-56db14d4bd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:52.182115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 797/1622] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:35:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:35:52.184157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3019bf3-7ed6-4c26-8f77-6f16d7c8a606 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:52.224531 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 826/1623] 10.222.0.43 () {58 vars in 1058 bytes} [Thu Jun 23 22:35:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:52.235848 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3019bf3-7ed6-4c26-8f77-6f16d7c8a606 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:52.237575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8b691-9eb3-4480-9aa9-56db14d4bd18 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:52.246891 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e979d64-1ad8-4b4f-abae-8711a97ca6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:52.247675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e979d64-1ad8-4b4f-abae-8711a97ca6d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:52.248413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e979d64-1ad8-4b4f-abae-8711a97ca6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:52.254070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e979d64-1ad8-4b4f-abae-8711a97ca6d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:52.254958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e979d64-1ad8-4b4f-abae-8711a97ca6d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:52.257098 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e979d64-1ad8-4b4f-abae-8711a97ca6d3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:52.267407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8b691-9eb3-4480-9aa9-56db14d4bd18 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:52.290041 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 798/1624] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:52.291029 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e979d64-1ad8-4b4f-abae-8711a97ca6d3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:52.301466 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052c18a-37fa-4a2b-bc2b-41490b557842 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:52.302146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052c18a-37fa-4a2b-bc2b-41490b557842 service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:52.302431 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052c18a-37fa-4a2b-bc2b-41490b557842 service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:52.302704 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052c18a-37fa-4a2b-bc2b-41490b557842 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:52.323995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e979d64-1ad8-4b4f-abae-8711a97ca6d3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:52.344582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd130942-521a-4d43-bcdd-13765f685935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:52.345478 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd130942-521a-4d43-bcdd-13765f685935 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:52.345767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd130942-521a-4d43-bcdd-13765f685935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:52.349325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd130942-521a-4d43-bcdd-13765f685935 admin admin] PATH_INFO: `/v3/projects/e9f392bbabb1414bad7685e6d65f6cdb/users/50a1f162eaf24c779f4b980279ddacb9/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:52.410615 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd130942-521a-4d43-bcdd-13765f685935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f392bbabb1414bad7685e6d65f6cdb, user_id=50a1f162eaf24c779f4b980279ddacb9, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:52.422109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd130942-521a-4d43-bcdd-13765f685935 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:52.513813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 799/1625] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:35:52 2022] PUT /identity/v3/projects/e9f392bbabb1414bad7685e6d65f6cdb/users/50a1f162eaf24c779f4b980279ddacb9/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:35:52.513813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd130942-521a-4d43-bcdd-13765f685935 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:52.627932 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052c18a-37fa-4a2b-bc2b-41490b557842 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:52.630205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052c18a-37fa-4a2b-bc2b-41490b557842 service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:52.657943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 827/1626] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:35:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:35:52.692641 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd130942-521a-4d43-bcdd-13765f685935 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:52.717525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2aab4-907d-4c55-85cc-e99ad012630b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:52.718228 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2aab4-907d-4c55-85cc-e99ad012630b admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:52.718521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2aab4-907d-4c55-85cc-e99ad012630b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:52.718809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2aab4-907d-4c55-85cc-e99ad012630b admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:52.722404 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e2aab4-907d-4c55-85cc-e99ad012630b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:52.724414 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e2aab4-907d-4c55-85cc-e99ad012630b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:52.737796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 800/1627] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:52.743391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052c18a-37fa-4a2b-bc2b-41490b557842 service nova] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:52.763670 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052c18a-37fa-4a2b-bc2b-41490b557842 service nova] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:52.779900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c0057-ae5c-444d-bf3b-c0324017373c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:52.782625 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c0057-ae5c-444d-bf3b-c0324017373c admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:52.785261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c0057-ae5c-444d-bf3b-c0324017373c admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:52.785854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c0057-ae5c-444d-bf3b-c0324017373c admin admin] PATH_INFO: `/v3/projects/e9f392bbabb1414bad7685e6d65f6cdb/users/50a1f162eaf24c779f4b980279ddacb9/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:52.793079 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c0057-ae5c-444d-bf3b-c0324017373c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f392bbabb1414bad7685e6d65f6cdb, user_id=50a1f162eaf24c779f4b980279ddacb9, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:52.803168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c0057-ae5c-444d-bf3b-c0324017373c admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:52.832382 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 828/1628] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:35:52 2022] PUT /identity/v3/projects/e9f392bbabb1414bad7685e6d65f6cdb/users/50a1f162eaf24c779f4b980279ddacb9/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:35:52.846556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6a070-5afd-4409-82d4-07e0a7ef96f3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:52.846947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6a070-5afd-4409-82d4-07e0a7ef96f3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:52.847580 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6a070-5afd-4409-82d4-07e0a7ef96f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:52.885770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03853695-97c6-4656-b2e2-d2e0992ec770 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:52.886080 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03853695-97c6-4656-b2e2-d2e0992ec770 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:52.886350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03853695-97c6-4656-b2e2-d2e0992ec770 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:52.889493 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 829/1629] 10.222.0.43 () {54 vars in 912 bytes} [Thu Jun 23 22:35:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:35:52.901366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecf277-715d-4fe0-b23f-021b8bd2e336 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:52.901610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecf277-715d-4fe0-b23f-021b8bd2e336 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:52.902396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecf277-715d-4fe0-b23f-021b8bd2e336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:52.914801 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d4f6a070-5afd-4409-82d4-07e0a7ef96f3 None None] MFA Rules not processed for user `50a1f162eaf24c779f4b980279ddacb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:52.916608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d4f6a070-5afd-4409-82d4-07e0a7ef96f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:52.943046 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-adecf277-715d-4fe0-b23f-021b8bd2e336 None None] MFA Rules not processed for user `35d686923ef541819de3c77847d38511`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:52.946752 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-adecf277-715d-4fe0-b23f-021b8bd2e336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:53.075466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 801/1630] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:53.089815 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6a070-5afd-4409-82d4-07e0a7ef96f3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:53.114040 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 830/1631] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 22:35:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:35:53.128012 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecf277-715d-4fe0-b23f-021b8bd2e336 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:53.180346 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:53.229934 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6a070-5afd-4409-82d4-07e0a7ef96f3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:53.237710 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ed131-a49d-4a1c-a175-124c1b3f8b5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:53.238377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ed131-a49d-4a1c-a175-124c1b3f8b5f service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:53.238671 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ed131-a49d-4a1c-a175-124c1b3f8b5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:53.238973 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ed131-a49d-4a1c-a175-124c1b3f8b5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:53.258579 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:53.260614 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ed131-a49d-4a1c-a175-124c1b3f8b5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:53.262266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ed131-a49d-4a1c-a175-124c1b3f8b5f service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:53.265518 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecf277-715d-4fe0-b23f-021b8bd2e336 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:53.272105 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c083e6-f990-4613-8f82-56373cf68a84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:53.273096 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c083e6-f990-4613-8f82-56373cf68a84 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:53.273646 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c083e6-f990-4613-8f82-56373cf68a84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:53.274151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c083e6-f990-4613-8f82-56373cf68a84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:53.289067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c083e6-f990-4613-8f82-56373cf68a84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:53.290974 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c083e6-f990-4613-8f82-56373cf68a84 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:53.295400 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 802/1632] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:35:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:35:53.306326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ea32f-df62-47ee-9446-af0731254b4d None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:53.306623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ea32f-df62-47ee-9446-af0731254b4d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:53.306908 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ea32f-df62-47ee-9446-af0731254b4d None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:53.309120 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 831/1633] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:35:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:35:53.317061 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 803/1634] 10.222.0.43 () {54 vars in 912 bytes} [Thu Jun 23 22:35:53 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:35:53.326567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa51a66-1bdf-497b-8cd9-2e2dae757c4e None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:53.326959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa51a66-1bdf-497b-8cd9-2e2dae757c4e None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:53.327366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa51a66-1bdf-497b-8cd9-2e2dae757c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:53.383833 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-4aa51a66-1bdf-497b-8cd9-2e2dae757c4e None None] MFA Rules not processed for user `35d686923ef541819de3c77847d38511`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:53.386813 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-4aa51a66-1bdf-497b-8cd9-2e2dae757c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:53.435939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 832/1635] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 22:35:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:53.502184 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ea32f-df62-47ee-9446-af0731254b4d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:53.514941 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5ea32f-df62-47ee-9446-af0731254b4d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:53.518775 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b2c1e-e92e-473d-9ccd-f9013f1f6c98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:53.520309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b2c1e-e92e-473d-9ccd-f9013f1f6c98 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:53.520309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b2c1e-e92e-473d-9ccd-f9013f1f6c98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:53.520309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b2c1e-e92e-473d-9ccd-f9013f1f6c98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:53.536664 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b2c1e-e92e-473d-9ccd-f9013f1f6c98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:53.538151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b2c1e-e92e-473d-9ccd-f9013f1f6c98 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:53.571581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 804/1636] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:35:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:35:54.562302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa51a66-1bdf-497b-8cd9-2e2dae757c4e None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:54.734270 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa51a66-1bdf-497b-8cd9-2e2dae757c4e None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:54.757773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913851d3-9f15-42c6-9cb8-a73835c0eec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:54.761940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913851d3-9f15-42c6-9cb8-a73835c0eec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:54.762874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913851d3-9f15-42c6-9cb8-a73835c0eec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:54.765087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913851d3-9f15-42c6-9cb8-a73835c0eec2 admin admin] PATH_INFO: `/v3/users/ea3c44869fe74d628b8dd5ecd0c22485` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:54.767737 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913851d3-9f15-42c6-9cb8-a73835c0eec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea3c44869fe74d628b8dd5ecd0c22485)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:54.770236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: WARNING py.warnings [None req-913851d3-9f15-42c6-9cb8-a73835c0eec2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:35:54.770236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: warnings.warn(msg) Jun 23 22:35:54.770236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: Jun 23 22:35:54.774361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913851d3-9f15-42c6-9cb8-a73835c0eec2 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:55.078641 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-913851d3-9f15-42c6-9cb8-a73835c0eec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea3c44869fe74d628b8dd5ecd0c22485'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:55.125090 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-913851d3-9f15-42c6-9cb8-a73835c0eec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea3c44869fe74d628b8dd5ecd0c22485'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:55.141471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-913851d3-9f15-42c6-9cb8-a73835c0eec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea3c44869fe74d628b8dd5ecd0c22485'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:55.171194 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-913851d3-9f15-42c6-9cb8-a73835c0eec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea3c44869fe74d628b8dd5ecd0c22485'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:55.172653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-913851d3-9f15-42c6-9cb8-a73835c0eec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea3c44869fe74d628b8dd5ecd0c22485'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:55.193702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 833/1637] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:35:54 2022] DELETE /identity/v3/users/ea3c44869fe74d628b8dd5ecd0c22485 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:35:55.195082 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0c20d-fec5-460b-bec6-4a2e56039277 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:55.195589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0c20d-fec5-460b-bec6-4a2e56039277 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:55.195924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0c20d-fec5-460b-bec6-4a2e56039277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:55.219764 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7fc0c20d-fec5-460b-bec6-4a2e56039277 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:55.221322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7fc0c20d-fec5-460b-bec6-4a2e56039277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:55.251053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913851d3-9f15-42c6-9cb8-a73835c0eec2 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:55.253095 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-913851d3-9f15-42c6-9cb8-a73835c0eec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:55.318374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 805/1638] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:55.372149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913851d3-9f15-42c6-9cb8-a73835c0eec2 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:55.386666 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb9b46-5272-4703-b1f8-2f9b2208f5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:55.387906 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb9b46-5272-4703-b1f8-2f9b2208f5d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:55.388231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb9b46-5272-4703-b1f8-2f9b2208f5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:55.388522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb9b46-5272-4703-b1f8-2f9b2208f5d0 admin admin] PATH_INFO: `/v3/users/ac0dc12e841345a482590080c487acfe` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:55.391272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb9b46-5272-4703-b1f8-2f9b2208f5d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac0dc12e841345a482590080c487acfe)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:55.393917 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb9b46-5272-4703-b1f8-2f9b2208f5d0 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:55.401708 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:55.408955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0c20d-fec5-460b-bec6-4a2e56039277 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:55.410696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7fc0c20d-fec5-460b-bec6-4a2e56039277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:55.519489 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c9bb9b46-5272-4703-b1f8-2f9b2208f5d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac0dc12e841345a482590080c487acfe'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:55.545336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c9bb9b46-5272-4703-b1f8-2f9b2208f5d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac0dc12e841345a482590080c487acfe'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:55.555685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0c20d-fec5-460b-bec6-4a2e56039277 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:55.562866 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c9bb9b46-5272-4703-b1f8-2f9b2208f5d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac0dc12e841345a482590080c487acfe'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:55.570589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c464c6-d810-4e59-b6d0-10ca1c783f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:55.571297 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c464c6-d810-4e59-b6d0-10ca1c783f04 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:55.575112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c464c6-d810-4e59-b6d0-10ca1c783f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:55.577574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c9bb9b46-5272-4703-b1f8-2f9b2208f5d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac0dc12e841345a482590080c487acfe'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:55.579027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c464c6-d810-4e59-b6d0-10ca1c783f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:55.579636 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c9bb9b46-5272-4703-b1f8-2f9b2208f5d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac0dc12e841345a482590080c487acfe'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:55.584046 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f7c464c6-d810-4e59-b6d0-10ca1c783f04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:55.599585 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 834/1639] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:35:55 2022] DELETE /identity/v3/users/ac0dc12e841345a482590080c487acfe => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:35:55.602058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a6d0a-92a1-4c8e-9d84-2449c1c6de88 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:55.603476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a6d0a-92a1-4c8e-9d84-2449c1c6de88 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:55.603884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a6d0a-92a1-4c8e-9d84-2449c1c6de88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:55.620141 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c464c6-d810-4e59-b6d0-10ca1c783f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:55.629499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c464c6-d810-4e59-b6d0-10ca1c783f04 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:55.652577 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 806/1640] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:35:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:35:55.653688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-d39a6d0a-92a1-4c8e-9d84-2449c1c6de88 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:55.655224 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d39a6d0a-92a1-4c8e-9d84-2449c1c6de88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:55.733397 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 835/1641] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:55.819045 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c464c6-d810-4e59-b6d0-10ca1c783f04 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:55.820654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f7c464c6-d810-4e59-b6d0-10ca1c783f04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:55.846970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c464c6-d810-4e59-b6d0-10ca1c783f04 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:56.045342 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bba118-56dd-4b9e-9249-120cc4dad39d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:56.046041 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bba118-56dd-4b9e-9249-120cc4dad39d service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:56.046378 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bba118-56dd-4b9e-9249-120cc4dad39d service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:56.046670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bba118-56dd-4b9e-9249-120cc4dad39d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:56.076563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bba118-56dd-4b9e-9249-120cc4dad39d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:56.084846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bba118-56dd-4b9e-9249-120cc4dad39d service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:56.112488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 807/1642] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:35:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:35:57.771797 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab55ac8-1d95-4f22-9bcd-16f393cd237a None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:57.772268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab55ac8-1d95-4f22-9bcd-16f393cd237a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:57.772362 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab55ac8-1d95-4f22-9bcd-16f393cd237a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:57.831280 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bba118-56dd-4b9e-9249-120cc4dad39d service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:57.833703 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-32bba118-56dd-4b9e-9249-120cc4dad39d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:57.839001 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-1ab55ac8-1d95-4f22-9bcd-16f393cd237a None None] MFA Rules not processed for user `f53284773f7a430db3d0e16d3c2cd262`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:57.846363 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1ab55ac8-1d95-4f22-9bcd-16f393cd237a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:57.866033 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bba118-56dd-4b9e-9249-120cc4dad39d service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:57.880372 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:57.881260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:57.881707 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:57.882212 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed admin admin] PATH_INFO: `/v3/projects/2d484d9303ed401fbd838f2c6bb9c04d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:57.883938 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d484d9303ed401fbd838f2c6bb9c04d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:57.885100 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: WARNING py.warnings [None req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 22:35:57.885100 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: warnings.warn(msg) Jun 23 22:35:57.885100 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: Jun 23 22:35:57.886744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:57.889445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:58.012915 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 836/1643] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:58.017468 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab55ac8-1d95-4f22-9bcd-16f393cd237a None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:58.022024 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1ab55ac8-1d95-4f22-9bcd-16f393cd237a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:58.030040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d484d9303ed401fbd838f2c6bb9c04d', 'request_id': 'req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:58.041866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d484d9303ed401fbd838f2c6bb9c04d', 'request_id': 'req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:58.049211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d484d9303ed401fbd838f2c6bb9c04d', 'request_id': 'req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:58.070128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab55ac8-1d95-4f22-9bcd-16f393cd237a None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:58.072300 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed admin admin] The token cache is being invalidate because project 2d484d9303ed401fbd838f2c6bb9c04d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:35:58.072607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:58.073285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:58.075354 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 808/1644] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:35:57 2022] DELETE /identity/v3/projects/2d484d9303ed401fbd838f2c6bb9c04d => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:35:58.077872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:58.080014 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:58.096670 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:58.221455 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d9ad7-d551-4d95-8be0-4727e7ae9a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:58.225443 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d9ad7-d551-4d95-8be0-4727e7ae9a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:58.225763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d9ad7-d551-4d95-8be0-4727e7ae9a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:58.226063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d9ad7-d551-4d95-8be0-4727e7ae9a8c admin admin] PATH_INFO: `/v3/users/4809dd1bd3ea4bb4a80de8a09f7eec6e` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:58.232377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d9ad7-d551-4d95-8be0-4727e7ae9a8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4809dd1bd3ea4bb4a80de8a09f7eec6e)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:58.234567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d9ad7-d551-4d95-8be0-4727e7ae9a8c admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:58.289282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c2899-70c7-4ac4-bf21-46eaacf8c2ed admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:58.302713 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f863467-760e-4302-a3d0-17db911d03b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:58.303644 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f863467-760e-4302-a3d0-17db911d03b7 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:58.303989 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f863467-760e-4302-a3d0-17db911d03b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:58.304362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f863467-760e-4302-a3d0-17db911d03b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:58.309926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9f863467-760e-4302-a3d0-17db911d03b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:58.350391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ec9d9ad7-d551-4d95-8be0-4727e7ae9a8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4809dd1bd3ea4bb4a80de8a09f7eec6e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:58.398048 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ec9d9ad7-d551-4d95-8be0-4727e7ae9a8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4809dd1bd3ea4bb4a80de8a09f7eec6e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:58.415967 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ec9d9ad7-d551-4d95-8be0-4727e7ae9a8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4809dd1bd3ea4bb4a80de8a09f7eec6e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:58.433101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ec9d9ad7-d551-4d95-8be0-4727e7ae9a8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4809dd1bd3ea4bb4a80de8a09f7eec6e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:58.435179 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ec9d9ad7-d551-4d95-8be0-4727e7ae9a8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4809dd1bd3ea4bb4a80de8a09f7eec6e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:58.455564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 837/1645] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:35:58 2022] DELETE /identity/v3/users/4809dd1bd3ea4bb4a80de8a09f7eec6e => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:35:58.459567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db540400-0032-4bfc-bd64-98e21b40beb4 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:58.461618 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db540400-0032-4bfc-bd64-98e21b40beb4 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:58.462274 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db540400-0032-4bfc-bd64-98e21b40beb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:58.494596 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f863467-760e-4302-a3d0-17db911d03b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:58.500286 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f863467-760e-4302-a3d0-17db911d03b7 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:58.508275 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-db540400-0032-4bfc-bd64-98e21b40beb4 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:58.510558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-db540400-0032-4bfc-bd64-98e21b40beb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:58.523840 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 809/1646] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:35:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:35:58.528204 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b89c0-69b7-43f8-a78a-f671465e4a45 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:58.530831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b89c0-69b7-43f8-a78a-f671465e4a45 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:58.531342 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b89c0-69b7-43f8-a78a-f671465e4a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:58.562019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-894b89c0-69b7-43f8-a78a-f671465e4a45 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:58.565395 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-894b89c0-69b7-43f8-a78a-f671465e4a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:58.637662 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:58.679650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 838/1647] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:58.680710 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db540400-0032-4bfc-bd64-98e21b40beb4 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:58.682226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-db540400-0032-4bfc-bd64-98e21b40beb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:58.704194 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db540400-0032-4bfc-bd64-98e21b40beb4 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:58.717769 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7258291-a7ff-42ff-a2fb-62ec17385e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:58.719372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7258291-a7ff-42ff-a2fb-62ec17385e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:58.719769 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7258291-a7ff-42ff-a2fb-62ec17385e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:58.720063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7258291-a7ff-42ff-a2fb-62ec17385e1e admin admin] PATH_INFO: `/v3/projects/5bdfd8b306f24510a91e99586724a2cb` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:58.722839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7258291-a7ff-42ff-a2fb-62ec17385e1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bdfd8b306f24510a91e99586724a2cb)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:58.727860 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7258291-a7ff-42ff-a2fb-62ec17385e1e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:58.765623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 810/1648] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:58.767304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47b36a-3e73-4b8e-9737-fb2544645980 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:58.770075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47b36a-3e73-4b8e-9737-fb2544645980 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:58.770534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47b36a-3e73-4b8e-9737-fb2544645980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:58.833591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6f47b36a-3e73-4b8e-9737-fb2544645980 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:58.836184 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:58.837181 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6f47b36a-3e73-4b8e-9737-fb2544645980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:58.892735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e7258291-a7ff-42ff-a2fb-62ec17385e1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bdfd8b306f24510a91e99586724a2cb', 'request_id': 'req-e7258291-a7ff-42ff-a2fb-62ec17385e1e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:58.899994 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e7258291-a7ff-42ff-a2fb-62ec17385e1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bdfd8b306f24510a91e99586724a2cb', 'request_id': 'req-e7258291-a7ff-42ff-a2fb-62ec17385e1e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:58.921795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e7258291-a7ff-42ff-a2fb-62ec17385e1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bdfd8b306f24510a91e99586724a2cb', 'request_id': 'req-e7258291-a7ff-42ff-a2fb-62ec17385e1e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:58.937166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e7258291-a7ff-42ff-a2fb-62ec17385e1e admin admin] The token cache is being invalidate because project 5bdfd8b306f24510a91e99586724a2cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:35:58.937783 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e7258291-a7ff-42ff-a2fb-62ec17385e1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:58.938616 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e7258291-a7ff-42ff-a2fb-62ec17385e1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:58.944823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 839/1649] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:35:58 2022] DELETE /identity/v3/projects/5bdfd8b306f24510a91e99586724a2cb => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:35:58.946104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7258291-a7ff-42ff-a2fb-62ec17385e1e admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:58.948349 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e7258291-a7ff-42ff-a2fb-62ec17385e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:58.980453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 811/1650] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:35:58.981375 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47b36a-3e73-4b8e-9737-fb2544645980 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:58.986751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6f47b36a-3e73-4b8e-9737-fb2544645980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:59.004856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47b36a-3e73-4b8e-9737-fb2544645980 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:59.011476 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd66a8-6ad1-4141-a20c-d4cd4b128665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:59.012382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd66a8-6ad1-4141-a20c-d4cd4b128665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:59.012669 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd66a8-6ad1-4141-a20c-d4cd4b128665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:59.012953 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd66a8-6ad1-4141-a20c-d4cd4b128665 admin admin] PATH_INFO: `/v3/users/50a1f162eaf24c779f4b980279ddacb9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:59.014294 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd66a8-6ad1-4141-a20c-d4cd4b128665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50a1f162eaf24c779f4b980279ddacb9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:59.016281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd66a8-6ad1-4141-a20c-d4cd4b128665 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:59.054951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7258291-a7ff-42ff-a2fb-62ec17385e1e admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:59.061043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d573e24-db22-4537-aebf-a2576a8a973e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:59.061965 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d573e24-db22-4537-aebf-a2576a8a973e service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:59.062327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d573e24-db22-4537-aebf-a2576a8a973e service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:59.062761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d573e24-db22-4537-aebf-a2576a8a973e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:59.065289 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8d573e24-db22-4537-aebf-a2576a8a973e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:59.084339 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d573e24-db22-4537-aebf-a2576a8a973e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:59.086125 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d573e24-db22-4537-aebf-a2576a8a973e service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:59.100477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 840/1651] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:35:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:35:59.101712 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d573e24-db22-4537-aebf-a2576a8a973e service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:59.113494 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d573e24-db22-4537-aebf-a2576a8a973e service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:59.115350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-44fd66a8-6ad1-4141-a20c-d4cd4b128665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50a1f162eaf24c779f4b980279ddacb9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:59.120438 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825be41-a984-42f5-b3fc-8e7e30e7b8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:59.121214 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825be41-a984-42f5-b3fc-8e7e30e7b8c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:59.121579 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825be41-a984-42f5-b3fc-8e7e30e7b8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:59.123125 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825be41-a984-42f5-b3fc-8e7e30e7b8c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:59.124095 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825be41-a984-42f5-b3fc-8e7e30e7b8c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:35:59.124479 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825be41-a984-42f5-b3fc-8e7e30e7b8c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:59.126256 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825be41-a984-42f5-b3fc-8e7e30e7b8c1 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:59.145988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 841/1652] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:35:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:35:59.150580 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825be41-a984-42f5-b3fc-8e7e30e7b8c1 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:59.151523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-44fd66a8-6ad1-4141-a20c-d4cd4b128665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50a1f162eaf24c779f4b980279ddacb9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:59.165309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-44fd66a8-6ad1-4141-a20c-d4cd4b128665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50a1f162eaf24c779f4b980279ddacb9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:59.166245 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-44fd66a8-6ad1-4141-a20c-d4cd4b128665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50a1f162eaf24c779f4b980279ddacb9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:59.167811 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-44fd66a8-6ad1-4141-a20c-d4cd4b128665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50a1f162eaf24c779f4b980279ddacb9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:35:59.168467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825be41-a984-42f5-b3fc-8e7e30e7b8c1 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:59.181112 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74a5b2-fd12-4f51-ab18-f710aeab2579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:59.182507 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74a5b2-fd12-4f51-ab18-f710aeab2579 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:59.182994 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74a5b2-fd12-4f51-ab18-f710aeab2579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:59.183392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da74a5b2-fd12-4f51-ab18-f710aeab2579 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:59.185794 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74a5b2-fd12-4f51-ab18-f710aeab2579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:59.188139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da74a5b2-fd12-4f51-ab18-f710aeab2579 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:59.196557 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 812/1653] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:35:58 2022] DELETE /identity/v3/users/50a1f162eaf24c779f4b980279ddacb9 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:35:59.250739 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd66a8-6ad1-4141-a20c-d4cd4b128665 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:59.251452 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 842/1654] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:35:59 2022] POST /identity/v3/projects => generated 401 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:59.254435 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-44fd66a8-6ad1-4141-a20c-d4cd4b128665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:35:59.435692 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:35:59.515856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd66a8-6ad1-4141-a20c-d4cd4b128665 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:59.622261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5b76b-2133-45fb-a454-7bf4cb920071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:59.624258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5b76b-2133-45fb-a454-7bf4cb920071 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:59.624819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5b76b-2133-45fb-a454-7bf4cb920071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:59.625317 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5b76b-2133-45fb-a454-7bf4cb920071 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:59.626326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5b76b-2133-45fb-a454-7bf4cb920071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:59.629637 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5b76b-2133-45fb-a454-7bf4cb920071 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:59.664740 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 813/1655] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:35:59 2022] POST /identity/v3/users => generated 301 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:35:59.670085 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74a5b2-fd12-4f51-ab18-f710aeab2579 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:59.690214 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da74a5b2-fd12-4f51-ab18-f710aeab2579 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:59.695503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28517c73-63e8-4175-bf71-7da9fe99c587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:59.696489 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28517c73-63e8-4175-bf71-7da9fe99c587 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:59.697069 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28517c73-63e8-4175-bf71-7da9fe99c587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:59.697453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28517c73-63e8-4175-bf71-7da9fe99c587 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:59.698201 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28517c73-63e8-4175-bf71-7da9fe99c587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:59.699960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28517c73-63e8-4175-bf71-7da9fe99c587 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:59.729492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5b76b-2133-45fb-a454-7bf4cb920071 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:59.743517 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 843/1656] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:59.747085 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5b76b-2133-45fb-a454-7bf4cb920071 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:59.755229 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a8655-9755-4be4-86a1-01dc2ba2f3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:59.756079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a8655-9755-4be4-86a1-01dc2ba2f3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:59.756440 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a8655-9755-4be4-86a1-01dc2ba2f3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:59.756789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a8655-9755-4be4-86a1-01dc2ba2f3d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:59.757613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a8655-9755-4be4-86a1-01dc2ba2f3d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:59.759396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a8655-9755-4be4-86a1-01dc2ba2f3d9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:59.777445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28517c73-63e8-4175-bf71-7da9fe99c587 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:35:59.787829 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 814/1657] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:35:59.802228 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28517c73-63e8-4175-bf71-7da9fe99c587 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:35:59.814705 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829c358-70af-484a-9279-845ad8313781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:35:59.817482 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829c358-70af-484a-9279-845ad8313781 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:59.817946 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829c358-70af-484a-9279-845ad8313781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:59.820687 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829c358-70af-484a-9279-845ad8313781 admin admin] PATH_INFO: `/v3/projects/19d2156f2b2d455da84804f4343aa70b/users/dce530dedb554c0db28edecdd2b0a41d/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:59.859331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829c358-70af-484a-9279-845ad8313781 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d2156f2b2d455da84804f4343aa70b, user_id=dce530dedb554c0db28edecdd2b0a41d, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:35:59.866160 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829c358-70af-484a-9279-845ad8313781 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:35:59.908729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 844/1658] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:35:59 2022] PUT /identity/v3/projects/19d2156f2b2d455da84804f4343aa70b/users/dce530dedb554c0db28edecdd2b0a41d/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:35:59.910274 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e3cd8-2cf8-4faa-a237-c0e020ab0802 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:35:59.910723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e3cd8-2cf8-4faa-a237-c0e020ab0802 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:35:59.911060 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e3cd8-2cf8-4faa-a237-c0e020ab0802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:35:59.987521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3f8e3cd8-2cf8-4faa-a237-c0e020ab0802 None None] MFA Rules not processed for user `dce530dedb554c0db28edecdd2b0a41d`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:35:59.988909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3f8e3cd8-2cf8-4faa-a237-c0e020ab0802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:00.066950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 815/1659] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:35:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:00.071246 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829c358-70af-484a-9279-845ad8313781 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:00.197232 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:00.224368 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829c358-70af-484a-9279-845ad8313781 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:00.240056 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd132f-3782-4f9f-b5f9-9ce82f186f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:00.242057 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd132f-3782-4f9f-b5f9-9ce82f186f81 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:00.242405 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd132f-3782-4f9f-b5f9-9ce82f186f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:00.243066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd132f-3782-4f9f-b5f9-9ce82f186f81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:00.243858 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd132f-3782-4f9f-b5f9-9ce82f186f81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:00.246880 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd132f-3782-4f9f-b5f9-9ce82f186f81 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:00.316314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 845/1660] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:00 2022] POST /identity/v3/projects => generated 401 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:00.316528 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e3cd8-2cf8-4faa-a237-c0e020ab0802 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:00.334126 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e3cd8-2cf8-4faa-a237-c0e020ab0802 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:00.496181 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd132f-3782-4f9f-b5f9-9ce82f186f81 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:00.499000 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-28bd132f-3782-4f9f-b5f9-9ce82f186f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:00.505807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:00.519166 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d16c8-c605-48a9-b980-1892ccd85149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:00.522198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d16c8-c605-48a9-b980-1892ccd85149 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:00.522198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d16c8-c605-48a9-b980-1892ccd85149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:00.522198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d16c8-c605-48a9-b980-1892ccd85149 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:00.522198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d16c8-c605-48a9-b980-1892ccd85149 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:00.524785 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d16c8-c605-48a9-b980-1892ccd85149 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:00.575209 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 816/1661] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:00 2022] POST /identity/v3/users => generated 307 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:00.579480 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d16c8-c605-48a9-b980-1892ccd85149 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:00.594910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d16c8-c605-48a9-b980-1892ccd85149 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:00.604363 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a80c5b-4d4c-4410-9702-db9f2506c65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:00.605141 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a80c5b-4d4c-4410-9702-db9f2506c65f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:00.605656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a80c5b-4d4c-4410-9702-db9f2506c65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:00.606036 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a80c5b-4d4c-4410-9702-db9f2506c65f admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:00.607959 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a80c5b-4d4c-4410-9702-db9f2506c65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:00.609911 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a80c5b-4d4c-4410-9702-db9f2506c65f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:00.636878 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 817/1662] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:00.639846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a80c5b-4d4c-4410-9702-db9f2506c65f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:00.674892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a80c5b-4d4c-4410-9702-db9f2506c65f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:00.683457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab58da-a877-45a1-8624-a22ac8acaacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:00.683764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd132f-3782-4f9f-b5f9-9ce82f186f81 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:00.687761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab58da-a877-45a1-8624-a22ac8acaacb admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:00.688128 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab58da-a877-45a1-8624-a22ac8acaacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:00.688629 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bab58da-a877-45a1-8624-a22ac8acaacb admin admin] PATH_INFO: `/v3/projects/707bb2b7fe514daaa9cbf57443e8d58d/users/f2d407e6f5fd4c1b8471508777478d3f/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:00.699624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa8f76-9616-42c7-a023-d207c2067b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:00.700521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa8f76-9616-42c7-a023-d207c2067b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:00.700844 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa8f76-9616-42c7-a023-d207c2067b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:00.701126 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa8f76-9616-42c7-a023-d207c2067b5a admin admin] PATH_INFO: `/v3/projects/e3870456e11b4171ab9ed057ebdf2088` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:00.709240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa8f76-9616-42c7-a023-d207c2067b5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3870456e11b4171ab9ed057ebdf2088)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:00.711941 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa8f76-9616-42c7-a023-d207c2067b5a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:00.712056 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab58da-a877-45a1-8624-a22ac8acaacb admin admin] RBAC: Authorizing `identity:create_grant(project_id=707bb2b7fe514daaa9cbf57443e8d58d, user_id=f2d407e6f5fd4c1b8471508777478d3f, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:00.715926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bab58da-a877-45a1-8624-a22ac8acaacb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:00.755372 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 818/1663] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:00 2022] PUT /identity/v3/projects/707bb2b7fe514daaa9cbf57443e8d58d/users/f2d407e6f5fd4c1b8471508777478d3f/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:00.759201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab58da-a877-45a1-8624-a22ac8acaacb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:00.851414 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-00aa8f76-9616-42c7-a023-d207c2067b5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3870456e11b4171ab9ed057ebdf2088', 'request_id': 'req-00aa8f76-9616-42c7-a023-d207c2067b5a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:00.868560 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-00aa8f76-9616-42c7-a023-d207c2067b5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3870456e11b4171ab9ed057ebdf2088', 'request_id': 'req-00aa8f76-9616-42c7-a023-d207c2067b5a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:00.886403 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-00aa8f76-9616-42c7-a023-d207c2067b5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3870456e11b4171ab9ed057ebdf2088', 'request_id': 'req-00aa8f76-9616-42c7-a023-d207c2067b5a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:00.911489 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-00aa8f76-9616-42c7-a023-d207c2067b5a admin admin] The token cache is being invalidate because project e3870456e11b4171ab9ed057ebdf2088 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:36:00.914000 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-00aa8f76-9616-42c7-a023-d207c2067b5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:00.917022 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-00aa8f76-9616-42c7-a023-d207c2067b5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:00.923102 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 846/1664] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:36:00 2022] DELETE /identity/v3/projects/e3870456e11b4171ab9ed057ebdf2088 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:00.928173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1feef36-b265-4ef1-ab67-8ca2b69fae45 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:00.928482 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1feef36-b265-4ef1-ab67-8ca2b69fae45 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:00.928773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1feef36-b265-4ef1-ab67-8ca2b69fae45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:00.970288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bab58da-a877-45a1-8624-a22ac8acaacb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:00.990193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-b1feef36-b265-4ef1-ab67-8ca2b69fae45 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:00.991886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b1feef36-b265-4ef1-ab67-8ca2b69fae45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:01.164364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d142549-450d-4a22-bc31-1adc205c0b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:01.165070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d142549-450d-4a22-bc31-1adc205c0b08 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:01.165360 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d142549-450d-4a22-bc31-1adc205c0b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:01.169982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d142549-450d-4a22-bc31-1adc205c0b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:01.170795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d142549-450d-4a22-bc31-1adc205c0b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:01.172472 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d142549-450d-4a22-bc31-1adc205c0b08 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:01.201675 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 847/1665] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:01.203346 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 819/1666] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:01.220402 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb2160-4100-46cb-a71a-72bac7905c14 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:01.221010 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d142549-450d-4a22-bc31-1adc205c0b08 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:01.221608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb2160-4100-46cb-a71a-72bac7905c14 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:01.221913 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb2160-4100-46cb-a71a-72bac7905c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:01.222782 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6d142549-450d-4a22-bc31-1adc205c0b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:01.237782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:01.245216 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d142549-450d-4a22-bc31-1adc205c0b08 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:01.259775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-3efb2160-4100-46cb-a71a-72bac7905c14 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:01.262477 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5e905-8ee0-4b94-abd6-112688a77d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:01.262671 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3efb2160-4100-46cb-a71a-72bac7905c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:01.264465 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5e905-8ee0-4b94-abd6-112688a77d8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:01.264935 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5e905-8ee0-4b94-abd6-112688a77d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:01.265481 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5e905-8ee0-4b94-abd6-112688a77d8d admin admin] PATH_INFO: `/v3/projects/707bb2b7fe514daaa9cbf57443e8d58d/users/f2d407e6f5fd4c1b8471508777478d3f/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:01.269561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5e905-8ee0-4b94-abd6-112688a77d8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=707bb2b7fe514daaa9cbf57443e8d58d, user_id=f2d407e6f5fd4c1b8471508777478d3f, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:01.276789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5e905-8ee0-4b94-abd6-112688a77d8d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:01.280154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 848/1667] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:01.281355 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb2160-4100-46cb-a71a-72bac7905c14 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:01.299591 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb2160-4100-46cb-a71a-72bac7905c14 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:01.308004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 820/1668] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:01 2022] PUT /identity/v3/projects/707bb2b7fe514daaa9cbf57443e8d58d/users/f2d407e6f5fd4c1b8471508777478d3f/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:01.323400 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a6139-c8ce-4419-92ea-e5c8b5112309 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:01.323793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a6139-c8ce-4419-92ea-e5c8b5112309 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:01.327168 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a6139-c8ce-4419-92ea-e5c8b5112309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:01.366739 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a40a6139-c8ce-4419-92ea-e5c8b5112309 None None] MFA Rules not processed for user `f2d407e6f5fd4c1b8471508777478d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:01.367944 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a40a6139-c8ce-4419-92ea-e5c8b5112309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:01.430192 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:01.512325 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95325c-e996-4d10-b9a6-13105c5e898e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:01.528133 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 821/1669] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:01 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:01.537378 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95325c-e996-4d10-b9a6-13105c5e898e admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:01.539339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a6139-c8ce-4419-92ea-e5c8b5112309 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:01.539474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95325c-e996-4d10-b9a6-13105c5e898e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:01.539740 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c95325c-e996-4d10-b9a6-13105c5e898e admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:01.540763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95325c-e996-4d10-b9a6-13105c5e898e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:36:01.541230 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a40a6139-c8ce-4419-92ea-e5c8b5112309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:01.545917 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95325c-e996-4d10-b9a6-13105c5e898e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:01.552772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c95325c-e996-4d10-b9a6-13105c5e898e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:01.592969 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 849/1670] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:36:01.597258 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95325c-e996-4d10-b9a6-13105c5e898e admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:01.636417 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c95325c-e996-4d10-b9a6-13105c5e898e admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:01.654528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4219a93-25d2-4bf4-82b6-46d040d00650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:01.655707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4219a93-25d2-4bf4-82b6-46d040d00650 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:01.658193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4219a93-25d2-4bf4-82b6-46d040d00650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:01.658671 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4219a93-25d2-4bf4-82b6-46d040d00650 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:01.659856 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4219a93-25d2-4bf4-82b6-46d040d00650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:01.664589 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4219a93-25d2-4bf4-82b6-46d040d00650 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:01.689403 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a6139-c8ce-4419-92ea-e5c8b5112309 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:01.714842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18344a1b-5105-49a7-8711-7dd3ae712e6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:01.715618 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18344a1b-5105-49a7-8711-7dd3ae712e6e service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:01.715932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18344a1b-5105-49a7-8711-7dd3ae712e6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:01.716323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18344a1b-5105-49a7-8711-7dd3ae712e6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:01.738571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 850/1671] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:01 2022] POST /identity/v3/projects => generated 409 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:01.738571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4219a93-25d2-4bf4-82b6-46d040d00650 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:01.742291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e4219a93-25d2-4bf4-82b6-46d040d00650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:01.762214 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18344a1b-5105-49a7-8711-7dd3ae712e6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:01.770267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18344a1b-5105-49a7-8711-7dd3ae712e6e service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:01.815394 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 822/1672] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:01 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:01.816462 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18344a1b-5105-49a7-8711-7dd3ae712e6e service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:01.840750 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18344a1b-5105-49a7-8711-7dd3ae712e6e service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:01.981653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4219a93-25d2-4bf4-82b6-46d040d00650 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:02.006507 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24579c6a-f97f-4afc-95d4-628585a5b70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:02.007725 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24579c6a-f97f-4afc-95d4-628585a5b70d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:02.008123 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24579c6a-f97f-4afc-95d4-628585a5b70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:02.008512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24579c6a-f97f-4afc-95d4-628585a5b70d admin admin] PATH_INFO: `/v3/projects/e9f392bbabb1414bad7685e6d65f6cdb` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:02.011952 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24579c6a-f97f-4afc-95d4-628585a5b70d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f392bbabb1414bad7685e6d65f6cdb)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:02.019150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb7af6-3779-4baf-82d1-8c86c591621b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:02.021886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24579c6a-f97f-4afc-95d4-628585a5b70d admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:02.022236 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb7af6-3779-4baf-82d1-8c86c591621b admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:02.022542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb7af6-3779-4baf-82d1-8c86c591621b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:02.022833 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb7af6-3779-4baf-82d1-8c86c591621b admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:02.023602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb7af6-3779-4baf-82d1-8c86c591621b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:02.026443 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb7af6-3779-4baf-82d1-8c86c591621b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:02.054370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 823/1673] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:01 2022] POST /identity/v3/users => generated 305 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:02.056147 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb7af6-3779-4baf-82d1-8c86c591621b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:02.176797 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-24579c6a-f97f-4afc-95d4-628585a5b70d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f392bbabb1414bad7685e6d65f6cdb', 'request_id': 'req-24579c6a-f97f-4afc-95d4-628585a5b70d'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:02.188925 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-24579c6a-f97f-4afc-95d4-628585a5b70d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f392bbabb1414bad7685e6d65f6cdb', 'request_id': 'req-24579c6a-f97f-4afc-95d4-628585a5b70d'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:02.211279 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-24579c6a-f97f-4afc-95d4-628585a5b70d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f392bbabb1414bad7685e6d65f6cdb', 'request_id': 'req-24579c6a-f97f-4afc-95d4-628585a5b70d'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:02.236003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:02.254828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-24579c6a-f97f-4afc-95d4-628585a5b70d admin admin] The token cache is being invalidate because project e9f392bbabb1414bad7685e6d65f6cdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:36:02.262447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-24579c6a-f97f-4afc-95d4-628585a5b70d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:02.263995 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-24579c6a-f97f-4afc-95d4-628585a5b70d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:02.267043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 851/1674] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:36:01 2022] DELETE /identity/v3/projects/e9f392bbabb1414bad7685e6d65f6cdb => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:02.268314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24579c6a-f97f-4afc-95d4-628585a5b70d admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:02.270077 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-24579c6a-f97f-4afc-95d4-628585a5b70d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:02.330628 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb7af6-3779-4baf-82d1-8c86c591621b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:02.354529 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:02.454614 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24579c6a-f97f-4afc-95d4-628585a5b70d admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:02.466562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba340a77-829f-48e2-8b45-f34863a174b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:02.467570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba340a77-829f-48e2-8b45-f34863a174b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:02.468045 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba340a77-829f-48e2-8b45-f34863a174b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:02.468380 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba340a77-829f-48e2-8b45-f34863a174b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:02.469211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba340a77-829f-48e2-8b45-f34863a174b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:02.472707 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62102172-cb9b-4318-9ebe-e21ea89af88c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:02.473390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62102172-cb9b-4318-9ebe-e21ea89af88c service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:02.473758 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62102172-cb9b-4318-9ebe-e21ea89af88c service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:02.474119 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62102172-cb9b-4318-9ebe-e21ea89af88c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:02.475814 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba340a77-829f-48e2-8b45-f34863a174b1 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:02.484835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-62102172-cb9b-4318-9ebe-e21ea89af88c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:02.510839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 852/1675] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:02.510839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585ce1a-f527-4237-964b-3be8f37dcf00 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:02.510839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585ce1a-f527-4237-964b-3be8f37dcf00 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:02.510839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585ce1a-f527-4237-964b-3be8f37dcf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:02.555240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-3585ce1a-f527-4237-964b-3be8f37dcf00 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:02.556784 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3585ce1a-f527-4237-964b-3be8f37dcf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:02.577558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 853/1676] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:02.578502 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585ce1a-f527-4237-964b-3be8f37dcf00 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:02.601685 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585ce1a-f527-4237-964b-3be8f37dcf00 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:02.614050 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713334e5-0360-4d98-b52f-769f723f7a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:02.614850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713334e5-0360-4d98-b52f-769f723f7a9e admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:02.615249 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713334e5-0360-4d98-b52f-769f723f7a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:02.615610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713334e5-0360-4d98-b52f-769f723f7a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:02.616447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713334e5-0360-4d98-b52f-769f723f7a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:02.619240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713334e5-0360-4d98-b52f-769f723f7a9e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:02.639867 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62102172-cb9b-4318-9ebe-e21ea89af88c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:02.641321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62102172-cb9b-4318-9ebe-e21ea89af88c service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:02.648420 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 854/1677] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:02.648420 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bb1e9-235b-40eb-8c1b-a4bb1665af5c None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:02.648420 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bb1e9-235b-40eb-8c1b-a4bb1665af5c None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:02.648420 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bb1e9-235b-40eb-8c1b-a4bb1665af5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:02.675385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 824/1678] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:02.676296 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62102172-cb9b-4318-9ebe-e21ea89af88c service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:02.691851 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-9b6bb1e9-235b-40eb-8c1b-a4bb1665af5c None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:02.693101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-9b6bb1e9-235b-40eb-8c1b-a4bb1665af5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:02.702459 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62102172-cb9b-4318-9ebe-e21ea89af88c service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:02.722943 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc128c1-9566-4d91-ba20-6f73e7628473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:02.723887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc128c1-9566-4d91-ba20-6f73e7628473 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:02.724175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc128c1-9566-4d91-ba20-6f73e7628473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:02.724457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc128c1-9566-4d91-ba20-6f73e7628473 admin admin] PATH_INFO: `/v3/projects/192b9fd67e49457290d6e89b8d0a2fb6/users/f04108f730bb430b9b8adb6181052e9a/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:02.725101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 855/1679] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:02.729430 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6bb1e9-235b-40eb-8c1b-a4bb1665af5c None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:02.761337 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6bb1e9-235b-40eb-8c1b-a4bb1665af5c None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:02.764405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc128c1-9566-4d91-ba20-6f73e7628473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=192b9fd67e49457290d6e89b8d0a2fb6, user_id=f04108f730bb430b9b8adb6181052e9a, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:02.767110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc128c1-9566-4d91-ba20-6f73e7628473 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:02.782658 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc8e91-55c9-4221-a0b3-2d9e3a982b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:02.783394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc8e91-55c9-4221-a0b3-2d9e3a982b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:02.783691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc8e91-55c9-4221-a0b3-2d9e3a982b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:02.783980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc8e91-55c9-4221-a0b3-2d9e3a982b44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:02.784946 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc8e91-55c9-4221-a0b3-2d9e3a982b44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:36:02.785260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc8e91-55c9-4221-a0b3-2d9e3a982b44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:02.787267 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc8e91-55c9-4221-a0b3-2d9e3a982b44 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:02.805241 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 825/1680] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:02 2022] PUT /identity/v3/projects/192b9fd67e49457290d6e89b8d0a2fb6/users/f04108f730bb430b9b8adb6181052e9a/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:02.813451 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 856/1681] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:36:02.817950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc128c1-9566-4d91-ba20-6f73e7628473 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:02.827224 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365309f-617f-4e3f-be7a-db7b7fc991e4 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:02.827536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365309f-617f-4e3f-be7a-db7b7fc991e4 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:02.827807 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365309f-617f-4e3f-be7a-db7b7fc991e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:02.949029 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-6365309f-617f-4e3f-be7a-db7b7fc991e4 None None] MFA Rules not processed for user `f04108f730bb430b9b8adb6181052e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:02.950039 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-6365309f-617f-4e3f-be7a-db7b7fc991e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:03.071573 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc128c1-9566-4d91-ba20-6f73e7628473 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:03.093113 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb222d2-a447-4aab-8f7f-28954d06a344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:03.095175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb222d2-a447-4aab-8f7f-28954d06a344 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:03.098201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb222d2-a447-4aab-8f7f-28954d06a344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:03.099515 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb222d2-a447-4aab-8f7f-28954d06a344 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:03.101852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb222d2-a447-4aab-8f7f-28954d06a344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:03.107719 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb222d2-a447-4aab-8f7f-28954d06a344 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:03.208288 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 857/1682] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:02 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:03.223514 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365309f-617f-4e3f-be7a-db7b7fc991e4 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:03.234832 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 826/1683] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:02 2022] POST /identity/v3/projects => generated 385 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:03.238110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb222d2-a447-4aab-8f7f-28954d06a344 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:03.258544 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365309f-617f-4e3f-be7a-db7b7fc991e4 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:03.379280 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:03.489108 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:03.531910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70ca7a-2108-4138-9b3e-f04f8c8837ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:03.534974 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70ca7a-2108-4138-9b3e-f04f8c8837ae admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:03.535248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70ca7a-2108-4138-9b3e-f04f8c8837ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:03.535545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70ca7a-2108-4138-9b3e-f04f8c8837ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:03.536255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70ca7a-2108-4138-9b3e-f04f8c8837ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:03.538287 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70ca7a-2108-4138-9b3e-f04f8c8837ae admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:03.550821 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb222d2-a447-4aab-8f7f-28954d06a344 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:03.579750 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af73b1-32c7-49f1-b895-6a17ce175d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:03.582601 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af73b1-32c7-49f1-b895-6a17ce175d77 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:03.582901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af73b1-32c7-49f1-b895-6a17ce175d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:03.583220 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af73b1-32c7-49f1-b895-6a17ce175d77 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:03.588355 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af73b1-32c7-49f1-b895-6a17ce175d77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:03.590868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af73b1-32c7-49f1-b895-6a17ce175d77 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:03.627786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 858/1684] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:03 2022] POST /identity/v3/projects => generated 407 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:03.632132 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70ca7a-2108-4138-9b3e-f04f8c8837ae admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:03.662948 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 827/1685] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:03 2022] POST /identity/v3/users => generated 293 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:03.662948 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af73b1-32c7-49f1-b895-6a17ce175d77 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:03.683275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af73b1-32c7-49f1-b895-6a17ce175d77 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:03.804569 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70ca7a-2108-4138-9b3e-f04f8c8837ae admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:03.808587 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ddf68a-0b6f-480b-9cfa-5f4bddea2912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:03.809372 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddf68a-0b6f-480b-9cfa-5f4bddea2912 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:03.809666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddf68a-0b6f-480b-9cfa-5f4bddea2912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:03.810598 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddf68a-0b6f-480b-9cfa-5f4bddea2912 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:03.811433 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ddf68a-0b6f-480b-9cfa-5f4bddea2912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:03.812975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ddf68a-0b6f-480b-9cfa-5f4bddea2912 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:03.824611 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 828/1686] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:36:03.830752 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ddf68a-0b6f-480b-9cfa-5f4bddea2912 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:03.831110 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e626a2e-27f8-4d88-9d17-6a8c3c087c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:03.832098 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e626a2e-27f8-4d88-9d17-6a8c3c087c36 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:03.832583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e626a2e-27f8-4d88-9d17-6a8c3c087c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:03.833049 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e626a2e-27f8-4d88-9d17-6a8c3c087c36 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:03.833947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e626a2e-27f8-4d88-9d17-6a8c3c087c36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:03.838388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e626a2e-27f8-4d88-9d17-6a8c3c087c36 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:03.841770 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ddf68a-0b6f-480b-9cfa-5f4bddea2912 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:03.850681 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964c8b7-ef9f-4121-a681-ac733b81fffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:03.851402 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964c8b7-ef9f-4121-a681-ac733b81fffa admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:03.851689 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964c8b7-ef9f-4121-a681-ac733b81fffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:03.851972 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964c8b7-ef9f-4121-a681-ac733b81fffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:03.852707 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964c8b7-ef9f-4121-a681-ac733b81fffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:03.854433 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964c8b7-ef9f-4121-a681-ac733b81fffa admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:03.870477 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 829/1687] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:03.875196 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964c8b7-ef9f-4121-a681-ac733b81fffa admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:03.887363 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964c8b7-ef9f-4121-a681-ac733b81fffa admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:03.890157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 859/1688] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:03 2022] POST /identity/v3/users => generated 310 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:03.892847 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e626a2e-27f8-4d88-9d17-6a8c3c087c36 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:03.896082 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330a851-399a-4161-82ed-bb2e774e48da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:03.898057 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330a851-399a-4161-82ed-bb2e774e48da admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:03.898601 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330a851-399a-4161-82ed-bb2e774e48da admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:03.899652 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330a851-399a-4161-82ed-bb2e774e48da admin admin] PATH_INFO: `/v3/projects/c6a8c615fcea43f79b97d91853b4ac3e/users/418fb2e877884959a2c027e8f742167f/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:03.918024 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e626a2e-27f8-4d88-9d17-6a8c3c087c36 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:03.924144 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330a851-399a-4161-82ed-bb2e774e48da admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a8c615fcea43f79b97d91853b4ac3e, user_id=418fb2e877884959a2c027e8f742167f, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:03.929148 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330a851-399a-4161-82ed-bb2e774e48da admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:03.932282 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf3dc5-d429-4ef6-ac88-65307748a2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:03.932707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf3dc5-d429-4ef6-ac88-65307748a2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:03.933158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf3dc5-d429-4ef6-ac88-65307748a2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:03.933419 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf3dc5-d429-4ef6-ac88-65307748a2f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:03.934203 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf3dc5-d429-4ef6-ac88-65307748a2f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:03.940601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf3dc5-d429-4ef6-ac88-65307748a2f2 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:03.951127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 830/1689] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:03 2022] PUT /identity/v3/projects/c6a8c615fcea43f79b97d91853b4ac3e/users/418fb2e877884959a2c027e8f742167f/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:36:03.961706 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f78aa5-9b60-4a1d-aafa-4ab648e1aa9d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:03.961706 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f78aa5-9b60-4a1d-aafa-4ab648e1aa9d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:03.961706 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f78aa5-9b60-4a1d-aafa-4ab648e1aa9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:03.968736 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 860/1690] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:03.969770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf3dc5-d429-4ef6-ac88-65307748a2f2 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:03.980385 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf3dc5-d429-4ef6-ac88-65307748a2f2 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:03.994800 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f5f78aa5-9b60-4a1d-aafa-4ab648e1aa9d None None] MFA Rules not processed for user `418fb2e877884959a2c027e8f742167f`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:03.996467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f5f78aa5-9b60-4a1d-aafa-4ab648e1aa9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:04.085886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec41d2-f181-4bcf-b554-c7cc7acbe839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:04.086804 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec41d2-f181-4bcf-b554-c7cc7acbe839 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:04.087116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec41d2-f181-4bcf-b554-c7cc7acbe839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:04.087398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec41d2-f181-4bcf-b554-c7cc7acbe839 admin admin] PATH_INFO: `/v3/projects/9d44ba037d414cb2b0471762e7d83141/users/baefb455c62c4844a56cc8f98d16287b/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:04.089973 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 831/1691] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:04.091046 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f78aa5-9b60-4a1d-aafa-4ab648e1aa9d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:04.092475 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f5f78aa5-9b60-4a1d-aafa-4ab648e1aa9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:04.106450 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f78aa5-9b60-4a1d-aafa-4ab648e1aa9d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:04.111900 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8cf9cf-07ab-43fc-90ae-d2725537f422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:04.112796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cf9cf-07ab-43fc-90ae-d2725537f422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:04.113080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cf9cf-07ab-43fc-90ae-d2725537f422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:04.113358 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cf9cf-07ab-43fc-90ae-d2725537f422 admin admin] PATH_INFO: `/v3/users/85f1409263c5421eaa55e6354761b0d0` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:04.114579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cf9cf-07ab-43fc-90ae-d2725537f422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85f1409263c5421eaa55e6354761b0d0)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:04.116578 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cf9cf-07ab-43fc-90ae-d2725537f422 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:04.119371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:04.162071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec41d2-f181-4bcf-b554-c7cc7acbe839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d44ba037d414cb2b0471762e7d83141, user_id=baefb455c62c4844a56cc8f98d16287b, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:04.164776 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec41d2-f181-4bcf-b554-c7cc7acbe839 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:04.205988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 861/1692] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:03 2022] PUT /identity/v3/projects/9d44ba037d414cb2b0471762e7d83141/users/baefb455c62c4844a56cc8f98d16287b/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:04.209205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec41d2-f181-4bcf-b554-c7cc7acbe839 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:04.211934 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-de8cf9cf-07ab-43fc-90ae-d2725537f422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85f1409263c5421eaa55e6354761b0d0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:04.223072 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-de8cf9cf-07ab-43fc-90ae-d2725537f422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85f1409263c5421eaa55e6354761b0d0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:04.235811 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-de8cf9cf-07ab-43fc-90ae-d2725537f422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85f1409263c5421eaa55e6354761b0d0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:04.237387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-de8cf9cf-07ab-43fc-90ae-d2725537f422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85f1409263c5421eaa55e6354761b0d0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:04.238371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-de8cf9cf-07ab-43fc-90ae-d2725537f422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85f1409263c5421eaa55e6354761b0d0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:04.273098 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 832/1693] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:36:04 2022] DELETE /identity/v3/users/85f1409263c5421eaa55e6354761b0d0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:04.278866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8cf9cf-07ab-43fc-90ae-d2725537f422 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:04.281449 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-de8cf9cf-07ab-43fc-90ae-d2725537f422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:04.321625 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:04.385620 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec41d2-f181-4bcf-b554-c7cc7acbe839 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:04.392235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8cf9cf-07ab-43fc-90ae-d2725537f422 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:04.398428 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc112c32-8f1d-48b0-be2e-251353a8fb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:04.398955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3426a-9344-4414-b263-904dbeebce19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:04.399662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3426a-9344-4414-b263-904dbeebce19 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:04.400117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc112c32-8f1d-48b0-be2e-251353a8fb05 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:04.400622 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3426a-9344-4414-b263-904dbeebce19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:04.400972 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3426a-9344-4414-b263-904dbeebce19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:04.401454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc112c32-8f1d-48b0-be2e-251353a8fb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:04.401694 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3426a-9344-4414-b263-904dbeebce19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:04.402136 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc112c32-8f1d-48b0-be2e-251353a8fb05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:04.403169 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc112c32-8f1d-48b0-be2e-251353a8fb05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:04.406171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc112c32-8f1d-48b0-be2e-251353a8fb05 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:04.407613 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3426a-9344-4414-b263-904dbeebce19 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:04.421452 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 833/1694] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:04.423425 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d732fd4-9f2b-409d-b962-0efbcdfaf8ef None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:04.423943 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d732fd4-9f2b-409d-b962-0efbcdfaf8ef None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:04.424477 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d732fd4-9f2b-409d-b962-0efbcdfaf8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:04.463946 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 862/1695] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:04 2022] POST /identity/v3/projects => generated 387 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:04.465410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3426a-9344-4414-b263-904dbeebce19 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:04.473396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3d732fd4-9f2b-409d-b962-0efbcdfaf8ef None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:04.474588 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3d732fd4-9f2b-409d-b962-0efbcdfaf8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:04.604517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 834/1696] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:04.605465 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d732fd4-9f2b-409d-b962-0efbcdfaf8ef None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:04.611551 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3d732fd4-9f2b-409d-b962-0efbcdfaf8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:04.635218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d732fd4-9f2b-409d-b962-0efbcdfaf8ef None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:04.647632 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3a81e-aaea-4a1b-9e05-044b5054c19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:04.648336 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3a81e-aaea-4a1b-9e05-044b5054c19e admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:04.648624 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3a81e-aaea-4a1b-9e05-044b5054c19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:04.648904 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3a81e-aaea-4a1b-9e05-044b5054c19e admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:04.649608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3a81e-aaea-4a1b-9e05-044b5054c19e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:04.658864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3a81e-aaea-4a1b-9e05-044b5054c19e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:04.665133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3426a-9344-4414-b263-904dbeebce19 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:04.672068 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a862eff-a7ee-45c7-8d12-578a9d5e3735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:04.673586 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a862eff-a7ee-45c7-8d12-578a9d5e3735 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:04.674454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a862eff-a7ee-45c7-8d12-578a9d5e3735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:04.674987 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a862eff-a7ee-45c7-8d12-578a9d5e3735 admin admin] PATH_INFO: `/v3/projects/9d44ba037d414cb2b0471762e7d83141/users/baefb455c62c4844a56cc8f98d16287b/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:04.679284 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a862eff-a7ee-45c7-8d12-578a9d5e3735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d44ba037d414cb2b0471762e7d83141, user_id=baefb455c62c4844a56cc8f98d16287b, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:04.685163 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a862eff-a7ee-45c7-8d12-578a9d5e3735 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:04.689262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:04.703413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 835/1697] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:04 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:04.708134 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3a81e-aaea-4a1b-9e05-044b5054c19e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:04.709417 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 863/1698] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:04 2022] PUT /identity/v3/projects/9d44ba037d414cb2b0471762e7d83141/users/baefb455c62c4844a56cc8f98d16287b/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:04.728255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3a81e-aaea-4a1b-9e05-044b5054c19e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:04.738418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855f616-3cad-46f8-8669-ae15a65c7e53 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:04.742553 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855f616-3cad-46f8-8669-ae15a65c7e53 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:04.742854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855f616-3cad-46f8-8669-ae15a65c7e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:04.820381 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd66013-0c12-44eb-9498-10b287247dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:04.821488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd66013-0c12-44eb-9498-10b287247dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:04.822195 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd66013-0c12-44eb-9498-10b287247dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:04.822713 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd66013-0c12-44eb-9498-10b287247dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:04.823990 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd66013-0c12-44eb-9498-10b287247dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:04.835673 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd66013-0c12-44eb-9498-10b287247dd7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:04.863033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-a855f616-3cad-46f8-8669-ae15a65c7e53 None None] MFA Rules not processed for user `baefb455c62c4844a56cc8f98d16287b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:04.863852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 836/1699] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:04.865505 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-a855f616-3cad-46f8-8669-ae15a65c7e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:04.866826 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd66013-0c12-44eb-9498-10b287247dd7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:04.875786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6fd66013-0c12-44eb-9498-10b287247dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:05.092774 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd66013-0c12-44eb-9498-10b287247dd7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:05.108314 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297e02b-3b6e-4897-ad2f-e120dcae258a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:05.108971 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297e02b-3b6e-4897-ad2f-e120dcae258a service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:05.109268 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297e02b-3b6e-4897-ad2f-e120dcae258a service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:05.109557 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297e02b-3b6e-4897-ad2f-e120dcae258a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:05.111743 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 864/1700] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:04 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:05.112675 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855f616-3cad-46f8-8669-ae15a65c7e53 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:05.147092 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297e02b-3b6e-4897-ad2f-e120dcae258a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:05.148563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297e02b-3b6e-4897-ad2f-e120dcae258a service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:05.152139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855f616-3cad-46f8-8669-ae15a65c7e53 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:05.171186 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04957c-8b1d-49af-8729-e0d3b69aed0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:05.171892 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04957c-8b1d-49af-8729-e0d3b69aed0c admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:05.172181 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04957c-8b1d-49af-8729-e0d3b69aed0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:05.172463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04957c-8b1d-49af-8729-e0d3b69aed0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:05.173265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04957c-8b1d-49af-8729-e0d3b69aed0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:05.174953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04957c-8b1d-49af-8729-e0d3b69aed0c admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:05.181885 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 837/1701] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:05.182607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297e02b-3b6e-4897-ad2f-e120dcae258a service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:05.199880 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e297e02b-3b6e-4897-ad2f-e120dcae258a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:05.222432 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 865/1702] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:05.223308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04957c-8b1d-49af-8729-e0d3b69aed0c admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:05.257912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04957c-8b1d-49af-8729-e0d3b69aed0c admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:05.297817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6919196-e5ab-47eb-bd14-16bad0e23caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:05.299856 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6919196-e5ab-47eb-bd14-16bad0e23caf admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:05.299856 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6919196-e5ab-47eb-bd14-16bad0e23caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:05.299856 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6919196-e5ab-47eb-bd14-16bad0e23caf admin admin] PATH_INFO: `/v3/projects/131d836799eb4701acd5ac2eaac0b85e/users/a7afffcffe034d2aad7ec6c00c4dc6ef/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:05.353695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6919196-e5ab-47eb-bd14-16bad0e23caf admin admin] RBAC: Authorizing `identity:create_grant(project_id=131d836799eb4701acd5ac2eaac0b85e, user_id=a7afffcffe034d2aad7ec6c00c4dc6ef, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:05.357657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6919196-e5ab-47eb-bd14-16bad0e23caf admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:05.398948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 866/1703] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:05 2022] PUT /identity/v3/projects/131d836799eb4701acd5ac2eaac0b85e/users/a7afffcffe034d2aad7ec6c00c4dc6ef/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:05.414174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3ef97-0e54-48a9-8de7-4330dd63755f None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:05.414915 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3ef97-0e54-48a9-8de7-4330dd63755f None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:05.414915 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3ef97-0e54-48a9-8de7-4330dd63755f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:05.417973 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297e02b-3b6e-4897-ad2f-e120dcae258a service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:05.427483 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:05.490997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-c3a3ef97-0e54-48a9-8de7-4330dd63755f None None] MFA Rules not processed for user `a7afffcffe034d2aad7ec6c00c4dc6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:05.492237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-c3a3ef97-0e54-48a9-8de7-4330dd63755f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:05.555873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339e63e-2968-47dc-a0d4-e02215e7b283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:05.556592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339e63e-2968-47dc-a0d4-e02215e7b283 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:05.556905 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339e63e-2968-47dc-a0d4-e02215e7b283 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:05.557121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339e63e-2968-47dc-a0d4-e02215e7b283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:05.582065 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:05.666451 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 867/1704] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:05.667989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3ef97-0e54-48a9-8de7-4330dd63755f None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:05.769781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6339e63e-2968-47dc-a0d4-e02215e7b283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:05.780738 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6339e63e-2968-47dc-a0d4-e02215e7b283 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:05.795465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3ef97-0e54-48a9-8de7-4330dd63755f None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:05.804902 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbead8e-000e-4937-b7b5-b4b19bcc20d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:05.805604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbead8e-000e-4937-b7b5-b4b19bcc20d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:05.806043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbead8e-000e-4937-b7b5-b4b19bcc20d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:05.806203 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbead8e-000e-4937-b7b5-b4b19bcc20d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:05.806917 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbead8e-000e-4937-b7b5-b4b19bcc20d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:05.808528 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 838/1705] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:05 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:05.819384 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbead8e-000e-4937-b7b5-b4b19bcc20d2 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:05.886020 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 868/1706] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:05 2022] POST /identity/v3/projects => generated 387 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:05.890991 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339e63e-2968-47dc-a0d4-e02215e7b283 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:05.912545 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339e63e-2968-47dc-a0d4-e02215e7b283 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:06.073771 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f59b80-b709-4891-8e55-74096edf742e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:06.076836 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f59b80-b709-4891-8e55-74096edf742e admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:06.077124 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f59b80-b709-4891-8e55-74096edf742e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:06.077410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f59b80-b709-4891-8e55-74096edf742e admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:06.078133 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f59b80-b709-4891-8e55-74096edf742e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:06.082330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f59b80-b709-4891-8e55-74096edf742e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:06.133517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 839/1707] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:05 2022] POST /identity/v3/users => generated 300 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:06.137472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbead8e-000e-4937-b7b5-b4b19bcc20d2 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:06.158415 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbead8e-000e-4937-b7b5-b4b19bcc20d2 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:06.189315 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7cc2fc-e440-4b60-a6e5-a751467452f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:06.197554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cc2fc-e440-4b60-a6e5-a751467452f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:06.197863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cc2fc-e440-4b60-a6e5-a751467452f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:06.198155 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7cc2fc-e440-4b60-a6e5-a751467452f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:06.198944 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7cc2fc-e440-4b60-a6e5-a751467452f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:06.210981 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7cc2fc-e440-4b60-a6e5-a751467452f5 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:06.246992 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f59b80-b709-4891-8e55-74096edf742e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:06.255640 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 869/1708] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:06.260537 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7cc2fc-e440-4b60-a6e5-a751467452f5 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:06.288536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7cc2fc-e440-4b60-a6e5-a751467452f5 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:06.302738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8aac6b-c55e-4991-8fab-9ed807de8236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:06.303768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8aac6b-c55e-4991-8fab-9ed807de8236 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:06.304069 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8aac6b-c55e-4991-8fab-9ed807de8236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:06.305846 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8aac6b-c55e-4991-8fab-9ed807de8236 admin admin] PATH_INFO: `/v3/projects/178847ac4e104fe98412714873a387f3/users/8d06a77800884a87969186ae5c21f938/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:06.351183 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8aac6b-c55e-4991-8fab-9ed807de8236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=178847ac4e104fe98412714873a387f3, user_id=8d06a77800884a87969186ae5c21f938, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:06.360813 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8aac6b-c55e-4991-8fab-9ed807de8236 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:06.392429 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f59b80-b709-4891-8e55-74096edf742e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:06.409359 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 870/1709] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:06 2022] PUT /identity/v3/projects/178847ac4e104fe98412714873a387f3/users/8d06a77800884a87969186ae5c21f938/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:06.413281 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2d12f-4de6-4a3a-b3b2-cf471b9eb6c9 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:06.414757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2d12f-4de6-4a3a-b3b2-cf471b9eb6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:06.415178 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2d12f-4de6-4a3a-b3b2-cf471b9eb6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:06.478897 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-7df2d12f-4de6-4a3a-b3b2-cf471b9eb6c9 None None] MFA Rules not processed for user `03486c7630314392b85271455d12aba2`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:06.483200 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7df2d12f-4de6-4a3a-b3b2-cf471b9eb6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:06.527899 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d3c42-680a-4c3f-8d05-8cfb3a781b57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:06.530128 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d3c42-680a-4c3f-8d05-8cfb3a781b57 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:06.530487 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d3c42-680a-4c3f-8d05-8cfb3a781b57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:06.530720 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d3c42-680a-4c3f-8d05-8cfb3a781b57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:06.536967 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-675d3c42-680a-4c3f-8d05-8cfb3a781b57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:06.551796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:06.588099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:06.663781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 871/1710] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:06.666032 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2d12f-4de6-4a3a-b3b2-cf471b9eb6c9 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:06.676748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d3c42-680a-4c3f-8d05-8cfb3a781b57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:06.679086 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d3c42-680a-4c3f-8d05-8cfb3a781b57 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:06.690430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 840/1711] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:06 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:36:06.691494 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d3c42-680a-4c3f-8d05-8cfb3a781b57 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:06.692913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-675d3c42-680a-4c3f-8d05-8cfb3a781b57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:06.912861 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2d12f-4de6-4a3a-b3b2-cf471b9eb6c9 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:06.924870 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d3c42-680a-4c3f-8d05-8cfb3a781b57 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:06.927306 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4371e-411a-4e04-9a1a-ccb4a6b78408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:06.928683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4371e-411a-4e04-9a1a-ccb4a6b78408 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:06.932528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4371e-411a-4e04-9a1a-ccb4a6b78408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:06.932946 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4371e-411a-4e04-9a1a-ccb4a6b78408 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:06.935176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4371e-411a-4e04-9a1a-ccb4a6b78408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:06.937250 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33d18b-eadd-4554-85a8-e3a4272e76aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:06.938300 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33d18b-eadd-4554-85a8-e3a4272e76aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:06.938726 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33d18b-eadd-4554-85a8-e3a4272e76aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:06.939171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33d18b-eadd-4554-85a8-e3a4272e76aa admin admin] PATH_INFO: `/v3/projects/6466e16d7f0d404db3f28d9e93b89b4a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:06.940864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4371e-411a-4e04-9a1a-ccb4a6b78408 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:06.942174 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33d18b-eadd-4554-85a8-e3a4272e76aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=6466e16d7f0d404db3f28d9e93b89b4a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:06.948896 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33d18b-eadd-4554-85a8-e3a4272e76aa admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:06.987740 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 872/1712] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:06.987740 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4371e-411a-4e04-9a1a-ccb4a6b78408 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:06.995723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-23b4371e-411a-4e04-9a1a-ccb4a6b78408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:07.090766 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ba33d18b-eadd-4554-85a8-e3a4272e76aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6466e16d7f0d404db3f28d9e93b89b4a', 'request_id': 'req-ba33d18b-eadd-4554-85a8-e3a4272e76aa'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:07.106084 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ba33d18b-eadd-4554-85a8-e3a4272e76aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6466e16d7f0d404db3f28d9e93b89b4a', 'request_id': 'req-ba33d18b-eadd-4554-85a8-e3a4272e76aa'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:07.117905 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ba33d18b-eadd-4554-85a8-e3a4272e76aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6466e16d7f0d404db3f28d9e93b89b4a', 'request_id': 'req-ba33d18b-eadd-4554-85a8-e3a4272e76aa'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:07.134434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ba33d18b-eadd-4554-85a8-e3a4272e76aa admin admin] The token cache is being invalidate because project 6466e16d7f0d404db3f28d9e93b89b4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:36:07.134988 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ba33d18b-eadd-4554-85a8-e3a4272e76aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:07.135776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ba33d18b-eadd-4554-85a8-e3a4272e76aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:07.163713 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 841/1713] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:36:06 2022] DELETE /identity/v3/projects/6466e16d7f0d404db3f28d9e93b89b4a => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:07.165822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33d18b-eadd-4554-85a8-e3a4272e76aa admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:07.167534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ba33d18b-eadd-4554-85a8-e3a4272e76aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:07.250644 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:07.344427 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4371e-411a-4e04-9a1a-ccb4a6b78408 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:07.345397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33d18b-eadd-4554-85a8-e3a4272e76aa admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:07.358230 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e96a8f-63d3-41ee-97c5-2fe84bea63c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:07.361110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e96a8f-63d3-41ee-97c5-2fe84bea63c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:07.361416 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e96a8f-63d3-41ee-97c5-2fe84bea63c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:07.361761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e96a8f-63d3-41ee-97c5-2fe84bea63c1 admin admin] PATH_INFO: `/v3/projects/178847ac4e104fe98412714873a387f3/users/8d06a77800884a87969186ae5c21f938/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:07.375608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e96a8f-63d3-41ee-97c5-2fe84bea63c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=178847ac4e104fe98412714873a387f3, user_id=8d06a77800884a87969186ae5c21f938, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:07.383949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e96a8f-63d3-41ee-97c5-2fe84bea63c1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:07.439351 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 842/1714] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:07 2022] PUT /identity/v3/projects/178847ac4e104fe98412714873a387f3/users/8d06a77800884a87969186ae5c21f938/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:07.448476 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cf600-14f5-42bb-892c-9ca471f4b4c0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:07.448476 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cf600-14f5-42bb-892c-9ca471f4b4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:07.448476 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cf600-14f5-42bb-892c-9ca471f4b4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:07.509417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-898cf600-14f5-42bb-892c-9ca471f4b4c0 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:07.510695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-898cf600-14f5-42bb-892c-9ca471f4b4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:07.568067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00250f3a-5645-4079-8b07-7ff00aeec02e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:07.568836 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00250f3a-5645-4079-8b07-7ff00aeec02e service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:07.569363 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00250f3a-5645-4079-8b07-7ff00aeec02e service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:07.569729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00250f3a-5645-4079-8b07-7ff00aeec02e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:07.572052 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-00250f3a-5645-4079-8b07-7ff00aeec02e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:07.597209 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:07.717254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 843/1715] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:07.726119 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25f149-5ebe-4033-9d4c-f4427ba7f531 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:07.726975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25f149-5ebe-4033-9d4c-f4427ba7f531 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:07.726975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25f149-5ebe-4033-9d4c-f4427ba7f531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:07.768322 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00250f3a-5645-4079-8b07-7ff00aeec02e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:07.771480 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00250f3a-5645-4079-8b07-7ff00aeec02e service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:07.776897 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4d25f149-5ebe-4033-9d4c-f4427ba7f531 None None] MFA Rules not processed for user `8d06a77800884a87969186ae5c21f938`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:07.781922 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4d25f149-5ebe-4033-9d4c-f4427ba7f531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:07.801828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 873/1716] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 816 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:07.803433 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e8cbb-fcbd-44cc-9e9a-7a5a830e0518 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:07.805522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e8cbb-fcbd-44cc-9e9a-7a5a830e0518 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:07.806468 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e8cbb-fcbd-44cc-9e9a-7a5a830e0518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:07.855779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-137e8cbb-fcbd-44cc-9e9a-7a5a830e0518 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:07.859845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-137e8cbb-fcbd-44cc-9e9a-7a5a830e0518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:07.882549 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 874/1717] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:07.884368 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e8cbb-fcbd-44cc-9e9a-7a5a830e0518 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:07.910360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e8cbb-fcbd-44cc-9e9a-7a5a830e0518 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:07.915236 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 844/1718] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:07.924630 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6247790-bdc5-4781-b8d1-627029401655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:07.925370 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6247790-bdc5-4781-b8d1-627029401655 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:07.925660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6247790-bdc5-4781-b8d1-627029401655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:07.930889 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25f149-5ebe-4033-9d4c-f4427ba7f531 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:07.932711 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4d25f149-5ebe-4033-9d4c-f4427ba7f531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:07.933585 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6247790-bdc5-4781-b8d1-627029401655 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:07.934460 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6247790-bdc5-4781-b8d1-627029401655 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:36:07.934794 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6247790-bdc5-4781-b8d1-627029401655 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:07.938033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6247790-bdc5-4781-b8d1-627029401655 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:07.957186 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 875/1719] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:36:07.961753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6247790-bdc5-4781-b8d1-627029401655 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:07.980036 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6247790-bdc5-4781-b8d1-627029401655 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:07.991811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b489350-2c34-445c-8221-8965eb8ca69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:07.994314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b489350-2c34-445c-8221-8965eb8ca69c admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:07.996410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b489350-2c34-445c-8221-8965eb8ca69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:07.996883 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b489350-2c34-445c-8221-8965eb8ca69c admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:08.000399 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b489350-2c34-445c-8221-8965eb8ca69c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:08.005921 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b489350-2c34-445c-8221-8965eb8ca69c admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:08.073429 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25f149-5ebe-4033-9d4c-f4427ba7f531 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:08.082325 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 876/1720] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:07 2022] POST /identity/v3/projects => generated 403 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:08.083327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b489350-2c34-445c-8221-8965eb8ca69c admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:08.090286 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:08.182543 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103fe21-04b0-4961-b60f-71d90dcf2abe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:08.184505 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103fe21-04b0-4961-b60f-71d90dcf2abe service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:08.185691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103fe21-04b0-4961-b60f-71d90dcf2abe service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:08.186516 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103fe21-04b0-4961-b60f-71d90dcf2abe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:08.278405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103fe21-04b0-4961-b60f-71d90dcf2abe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:08.280430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3103fe21-04b0-4961-b60f-71d90dcf2abe service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:08.293120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 845/1721] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:36:08.311571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b489350-2c34-445c-8221-8965eb8ca69c admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:08.342388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a141fb-0463-40c1-bd2a-bb4671c9a65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:08.347187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a141fb-0463-40c1-bd2a-bb4671c9a65e admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:08.347809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a141fb-0463-40c1-bd2a-bb4671c9a65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:08.347809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a141fb-0463-40c1-bd2a-bb4671c9a65e admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:08.348553 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a141fb-0463-40c1-bd2a-bb4671c9a65e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:08.375200 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a141fb-0463-40c1-bd2a-bb4671c9a65e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:08.434471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 877/1722] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:08 2022] POST /identity/v3/users => generated 302 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:08.436363 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103fe21-04b0-4961-b60f-71d90dcf2abe service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:08.454885 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103fe21-04b0-4961-b60f-71d90dcf2abe service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:08.469350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132a94d-cc5e-4cbd-90e3-ceb27b9e7334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:08.470324 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132a94d-cc5e-4cbd-90e3-ceb27b9e7334 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:08.470717 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132a94d-cc5e-4cbd-90e3-ceb27b9e7334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:08.471304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132a94d-cc5e-4cbd-90e3-ceb27b9e7334 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:08.472339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132a94d-cc5e-4cbd-90e3-ceb27b9e7334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:08.476358 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132a94d-cc5e-4cbd-90e3-ceb27b9e7334 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:08.503909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 846/1723] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:08.506325 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a141fb-0463-40c1-bd2a-bb4671c9a65e admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:08.527044 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a141fb-0463-40c1-bd2a-bb4671c9a65e admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:08.548175 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7deb4-e90c-408e-9777-995dfec69b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:08.548878 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7deb4-e90c-408e-9777-995dfec69b5a admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:08.549161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7deb4-e90c-408e-9777-995dfec69b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:08.549425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7deb4-e90c-408e-9777-995dfec69b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:08.550187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7deb4-e90c-408e-9777-995dfec69b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:08.558374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7deb4-e90c-408e-9777-995dfec69b5a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:08.579180 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132a94d-cc5e-4cbd-90e3-ceb27b9e7334 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:08.581178 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 878/1724] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:08.593312 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132a94d-cc5e-4cbd-90e3-ceb27b9e7334 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:08.607853 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee1e91-3f58-411c-91a6-37f7971d6fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:08.611445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee1e91-3f58-411c-91a6-37f7971d6fd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:08.611779 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee1e91-3f58-411c-91a6-37f7971d6fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:08.612240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee1e91-3f58-411c-91a6-37f7971d6fd2 admin admin] PATH_INFO: `/v3/projects/d43bc11b97484a04bb667891d05a3411/users/01468d30f7c74480a9eeab7fcaec5889/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:08.663663 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee1e91-3f58-411c-91a6-37f7971d6fd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d43bc11b97484a04bb667891d05a3411, user_id=01468d30f7c74480a9eeab7fcaec5889, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:08.673389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee1e91-3f58-411c-91a6-37f7971d6fd2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:08.721911 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 847/1725] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:08 2022] PUT /identity/v3/projects/d43bc11b97484a04bb667891d05a3411/users/01468d30f7c74480a9eeab7fcaec5889/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:08.732729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7deb4-e90c-408e-9777-995dfec69b5a admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:08.733588 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d1461-e829-43ed-9457-456c47bec2c6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:08.734412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d1461-e829-43ed-9457-456c47bec2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:08.735055 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d1461-e829-43ed-9457-456c47bec2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:08.742816 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-09f7deb4-e90c-408e-9777-995dfec69b5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:08.768665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7deb4-e90c-408e-9777-995dfec69b5a admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:08.813174 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-db8d1461-e829-43ed-9457-456c47bec2c6 None None] MFA Rules not processed for user `01468d30f7c74480a9eeab7fcaec5889`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:08.814460 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-db8d1461-e829-43ed-9457-456c47bec2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:08.920292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:08.983816 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496cafa6-a708-41e9-a9cb-e704236fe5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:08.985238 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cafa6-a708-41e9-a9cb-e704236fe5f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:08.985790 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cafa6-a708-41e9-a9cb-e704236fe5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:08.986683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cafa6-a708-41e9-a9cb-e704236fe5f2 admin admin] PATH_INFO: `/v3/users/dce530dedb554c0db28edecdd2b0a41d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:08.993189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496cafa6-a708-41e9-a9cb-e704236fe5f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dce530dedb554c0db28edecdd2b0a41d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:09.000141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496cafa6-a708-41e9-a9cb-e704236fe5f2 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:09.100889 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 848/1726] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:09.108558 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd72bd-e495-46fe-afc2-4953241bc71a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:09.109408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd72bd-e495-46fe-afc2-4953241bc71a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:09.113554 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd72bd-e495-46fe-afc2-4953241bc71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:09.188793 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-496cafa6-a708-41e9-a9cb-e704236fe5f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dce530dedb554c0db28edecdd2b0a41d'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:09.215047 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9ecd72bd-e495-46fe-afc2-4953241bc71a None None] MFA Rules not processed for user `f53284773f7a430db3d0e16d3c2cd262`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:09.221154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9ecd72bd-e495-46fe-afc2-4953241bc71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:09.253380 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-496cafa6-a708-41e9-a9cb-e704236fe5f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dce530dedb554c0db28edecdd2b0a41d'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:09.254463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:09.280224 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-496cafa6-a708-41e9-a9cb-e704236fe5f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dce530dedb554c0db28edecdd2b0a41d'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:09.299879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-496cafa6-a708-41e9-a9cb-e704236fe5f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dce530dedb554c0db28edecdd2b0a41d'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:09.300671 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-496cafa6-a708-41e9-a9cb-e704236fe5f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dce530dedb554c0db28edecdd2b0a41d'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:09.334309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 879/1727] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:36:08 2022] DELETE /identity/v3/users/dce530dedb554c0db28edecdd2b0a41d => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:09.334309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496cafa6-a708-41e9-a9cb-e704236fe5f2 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:09.334309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-496cafa6-a708-41e9-a9cb-e704236fe5f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:09.497921 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 849/1728] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:09.499374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68098738-6d7a-4ba8-a3dc-22f47f02971a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:09.499670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68098738-6d7a-4ba8-a3dc-22f47f02971a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:09.499998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68098738-6d7a-4ba8-a3dc-22f47f02971a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:09.562782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496cafa6-a708-41e9-a9cb-e704236fe5f2 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:09.572862 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd92822-947a-49bc-afe4-b660a1369f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:09.574088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd92822-947a-49bc-afe4-b660a1369f75 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:09.574542 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd92822-947a-49bc-afe4-b660a1369f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:09.574744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-68098738-6d7a-4ba8-a3dc-22f47f02971a None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:09.575166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd92822-947a-49bc-afe4-b660a1369f75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:09.576639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd92822-947a-49bc-afe4-b660a1369f75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:09.580743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-68098738-6d7a-4ba8-a3dc-22f47f02971a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:09.585574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd92822-947a-49bc-afe4-b660a1369f75 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:09.594788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 850/1729] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:09.674500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68098738-6d7a-4ba8-a3dc-22f47f02971a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:09.676899 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-68098738-6d7a-4ba8-a3dc-22f47f02971a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:09.709764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 880/1730] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:09 2022] POST /identity/v3/projects => generated 405 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:09.717444 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd92822-947a-49bc-afe4-b660a1369f75 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:09.859730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68098738-6d7a-4ba8-a3dc-22f47f02971a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:09.887740 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:09.929674 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd92822-947a-49bc-afe4-b660a1369f75 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:09.946718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc41cf-9d5a-42c6-91a1-99511a6c3401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:09.947647 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc41cf-9d5a-42c6-91a1-99511a6c3401 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:09.947935 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc41cf-9d5a-42c6-91a1-99511a6c3401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:09.948273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc41cf-9d5a-42c6-91a1-99511a6c3401 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:09.949104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc41cf-9d5a-42c6-91a1-99511a6c3401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:09.951536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc41cf-9d5a-42c6-91a1-99511a6c3401 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:10.014879 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6c633-b0f2-45f5-a241-24d6655ba986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:10.015923 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6c633-b0f2-45f5-a241-24d6655ba986 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:10.017354 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6c633-b0f2-45f5-a241-24d6655ba986 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:10.017878 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6c633-b0f2-45f5-a241-24d6655ba986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:10.023343 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 881/1731] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:09 2022] POST /identity/v3/users => generated 309 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:10.024446 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc41cf-9d5a-42c6-91a1-99511a6c3401 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:10.026502 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-71bc41cf-9d5a-42c6-91a1-99511a6c3401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:10.035506 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6c633-b0f2-45f5-a241-24d6655ba986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:10.037371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6c633-b0f2-45f5-a241-24d6655ba986 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:10.054665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 851/1732] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:10.056253 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6c633-b0f2-45f5-a241-24d6655ba986 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:10.069450 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6c633-b0f2-45f5-a241-24d6655ba986 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:10.075096 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a3b39-6dee-49b9-b6e1-7cad9741322e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:10.076885 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a3b39-6dee-49b9-b6e1-7cad9741322e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:10.077388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a3b39-6dee-49b9-b6e1-7cad9741322e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:10.077879 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a3b39-6dee-49b9-b6e1-7cad9741322e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:10.078840 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a3b39-6dee-49b9-b6e1-7cad9741322e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:10.081414 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a3b39-6dee-49b9-b6e1-7cad9741322e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:10.097426 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 852/1733] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:10.101338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a3b39-6dee-49b9-b6e1-7cad9741322e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:10.154548 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a3b39-6dee-49b9-b6e1-7cad9741322e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:10.162371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc41cf-9d5a-42c6-91a1-99511a6c3401 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:10.168278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62ce8f-0d4a-4db3-8347-3dbee2e26322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:10.171456 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62ce8f-0d4a-4db3-8347-3dbee2e26322 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:10.172786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2e312-0d25-42f5-a6bb-d68b3f050add service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:10.173477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2e312-0d25-42f5-a6bb-d68b3f050add service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:10.173764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2e312-0d25-42f5-a6bb-d68b3f050add service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:10.174047 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2e312-0d25-42f5-a6bb-d68b3f050add service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:10.174162 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62ce8f-0d4a-4db3-8347-3dbee2e26322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:10.175974 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62ce8f-0d4a-4db3-8347-3dbee2e26322 admin admin] PATH_INFO: `/v3/projects/3617c63403f349bb884c08d9a0fb0cf5/users/51d89d2745a14aac99f5b971816845b3/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:10.182515 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e5f2e312-0d25-42f5-a6bb-d68b3f050add service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:10.227941 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62ce8f-0d4a-4db3-8347-3dbee2e26322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3617c63403f349bb884c08d9a0fb0cf5, user_id=51d89d2745a14aac99f5b971816845b3, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:10.230886 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62ce8f-0d4a-4db3-8347-3dbee2e26322 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:10.271325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 853/1734] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:10 2022] PUT /identity/v3/projects/3617c63403f349bb884c08d9a0fb0cf5/users/51d89d2745a14aac99f5b971816845b3/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:10.275980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62ce8f-0d4a-4db3-8347-3dbee2e26322 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:10.354082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2e312-0d25-42f5-a6bb-d68b3f050add service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:10.355611 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2e312-0d25-42f5-a6bb-d68b3f050add service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:10.370954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:10.374060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 882/1735] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:10.455734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62ce8f-0d4a-4db3-8347-3dbee2e26322 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:10.471652 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d6897-2a06-47ea-9033-e69dc9a10124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:10.472369 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d6897-2a06-47ea-9033-e69dc9a10124 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:10.472661 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d6897-2a06-47ea-9033-e69dc9a10124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:10.472947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d6897-2a06-47ea-9033-e69dc9a10124 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:10.478490 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d6897-2a06-47ea-9033-e69dc9a10124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:10.483597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d6897-2a06-47ea-9033-e69dc9a10124 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:10.509937 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2e312-0d25-42f5-a6bb-d68b3f050add service cinder] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:10.514867 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 854/1736] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:10.531866 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2e312-0d25-42f5-a6bb-d68b3f050add service cinder] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:10.556094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd68add-85ec-4df4-90d9-b1f402da8eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:10.562218 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd68add-85ec-4df4-90d9-b1f402da8eac admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:10.562515 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd68add-85ec-4df4-90d9-b1f402da8eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:10.564674 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd68add-85ec-4df4-90d9-b1f402da8eac admin admin] PATH_INFO: `/v3/projects/3617c63403f349bb884c08d9a0fb0cf5/users/51d89d2745a14aac99f5b971816845b3/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:10.571075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd68add-85ec-4df4-90d9-b1f402da8eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=3617c63403f349bb884c08d9a0fb0cf5, user_id=51d89d2745a14aac99f5b971816845b3, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:10.587193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd68add-85ec-4df4-90d9-b1f402da8eac admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:10.648639 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274ab42-949f-4cb4-9249-5cba55dfacc0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:10.649207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274ab42-949f-4cb4-9249-5cba55dfacc0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:10.649699 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274ab42-949f-4cb4-9249-5cba55dfacc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:10.672520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 883/1737] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:10 2022] PUT /identity/v3/projects/3617c63403f349bb884c08d9a0fb0cf5/users/51d89d2745a14aac99f5b971816845b3/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:10.674576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5bad9-9795-4c7c-b68d-c86579257dd6 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:10.674877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5bad9-9795-4c7c-b68d-c86579257dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:10.678106 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5bad9-9795-4c7c-b68d-c86579257dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:10.742697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8274ab42-949f-4cb4-9249-5cba55dfacc0 None None] MFA Rules not processed for user `dc246e144afc46698000f7f53ef6489e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:10.744111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8274ab42-949f-4cb4-9249-5cba55dfacc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:10.824230 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-7fb5bad9-9795-4c7c-b68d-c86579257dd6 None None] MFA Rules not processed for user `51d89d2745a14aac99f5b971816845b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:10.826139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7fb5bad9-9795-4c7c-b68d-c86579257dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:10.949392 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 855/1738] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:36:10.952023 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797c9f8-b4e0-4355-948f-e843380ebb23 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:10.952300 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797c9f8-b4e0-4355-948f-e843380ebb23 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:10.952588 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797c9f8-b4e0-4355-948f-e843380ebb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:11.001583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3797c9f8-b4e0-4355-948f-e843380ebb23 None None] MFA Rules not processed for user `dc246e144afc46698000f7f53ef6489e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:11.003080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3797c9f8-b4e0-4355-948f-e843380ebb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:11.023441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 856/1739] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:36:11.037768 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797c9f8-b4e0-4355-948f-e843380ebb23 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:11.039437 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3797c9f8-b4e0-4355-948f-e843380ebb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:11.081538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 884/1740] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:10 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:11.087039 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5bad9-9795-4c7c-b68d-c86579257dd6 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:11.090204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7fb5bad9-9795-4c7c-b68d-c86579257dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:11.102462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:11.316776 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5bad9-9795-4c7c-b68d-c86579257dd6 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:11.325871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e1d9e-cbc1-431f-984c-f962ccc1f3d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:11.326562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e1d9e-cbc1-431f-984c-f962ccc1f3d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:11.326846 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e1d9e-cbc1-431f-984c-f962ccc1f3d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:11.327181 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e1d9e-cbc1-431f-984c-f962ccc1f3d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:11.330186 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797c9f8-b4e0-4355-948f-e843380ebb23 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:11.346187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66792a4b-a949-48d5-af2d-20726624c8b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:11.347300 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66792a4b-a949-48d5-af2d-20726624c8b1 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:11.348865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66792a4b-a949-48d5-af2d-20726624c8b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:11.348865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66792a4b-a949-48d5-af2d-20726624c8b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:11.351713 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e1d9e-cbc1-431f-984c-f962ccc1f3d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:11.354254 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e1d9e-cbc1-431f-984c-f962ccc1f3d7 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:11.373627 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66792a4b-a949-48d5-af2d-20726624c8b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:11.375618 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 885/1741] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:11 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:36:11.380809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66792a4b-a949-48d5-af2d-20726624c8b1 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:11.410330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 857/1742] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:36:11.771953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d515dcb-b443-4f72-851e-ab3431bb7c0c None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:11.776557 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d515dcb-b443-4f72-851e-ab3431bb7c0c None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:11.776884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d515dcb-b443-4f72-851e-ab3431bb7c0c None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:11.778576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 886/1743] 10.222.0.43 () {54 vars in 912 bytes} [Thu Jun 23 22:36:11 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:36:11.788908 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184173ce-dfb7-4d71-95a1-32b4699b4969 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:11.789216 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184173ce-dfb7-4d71-95a1-32b4699b4969 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:11.792006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184173ce-dfb7-4d71-95a1-32b4699b4969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:11.888906 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-184173ce-dfb7-4d71-95a1-32b4699b4969 None None] MFA Rules not processed for user `37ba527b96634fc2a48b6ca9597d8191`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:11.890449 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-184173ce-dfb7-4d71-95a1-32b4699b4969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:11.929058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:11.984037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d515dcb-b443-4f72-851e-ab3431bb7c0c None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:11.986852 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-0d515dcb-b443-4f72-851e-ab3431bb7c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:12.008987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 858/1744] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 22:36:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:12.056469 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184173ce-dfb7-4d71-95a1-32b4699b4969 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:12.070981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184173ce-dfb7-4d71-95a1-32b4699b4969 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:12.077337 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05cd15-59aa-428f-ae3c-81ae38bfcaac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:12.078018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05cd15-59aa-428f-ae3c-81ae38bfcaac service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:12.078312 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05cd15-59aa-428f-ae3c-81ae38bfcaac service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:12.078604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05cd15-59aa-428f-ae3c-81ae38bfcaac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:12.101493 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05cd15-59aa-428f-ae3c-81ae38bfcaac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:12.103193 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05cd15-59aa-428f-ae3c-81ae38bfcaac service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:12.138647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 859/1745] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:36:12.140066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d515dcb-b443-4f72-851e-ab3431bb7c0c None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:12.143096 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25191a5f-14a3-4ccf-9d40-518a5d593b7b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:12.143425 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25191a5f-14a3-4ccf-9d40-518a5d593b7b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:12.143757 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25191a5f-14a3-4ccf-9d40-518a5d593b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:12.146510 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75120847-5325-4963-b97d-34bc85776081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:12.148451 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75120847-5325-4963-b97d-34bc85776081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:12.148451 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75120847-5325-4963-b97d-34bc85776081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:12.148451 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75120847-5325-4963-b97d-34bc85776081 admin admin] PATH_INFO: `/v3/projects/19d2156f2b2d455da84804f4343aa70b` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:12.150150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75120847-5325-4963-b97d-34bc85776081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19d2156f2b2d455da84804f4343aa70b)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:12.152393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75120847-5325-4963-b97d-34bc85776081 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:12.185312 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-25191a5f-14a3-4ccf-9d40-518a5d593b7b None None] MFA Rules not processed for user `53c519b28b174ef385dd7e76b85cf504`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:12.186692 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-25191a5f-14a3-4ccf-9d40-518a5d593b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:12.255304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-75120847-5325-4963-b97d-34bc85776081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19d2156f2b2d455da84804f4343aa70b', 'request_id': 'req-75120847-5325-4963-b97d-34bc85776081'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:12.267935 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-75120847-5325-4963-b97d-34bc85776081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19d2156f2b2d455da84804f4343aa70b', 'request_id': 'req-75120847-5325-4963-b97d-34bc85776081'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:12.277172 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-75120847-5325-4963-b97d-34bc85776081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19d2156f2b2d455da84804f4343aa70b', 'request_id': 'req-75120847-5325-4963-b97d-34bc85776081'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:12.290124 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 860/1746] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:12.291695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fbddd-16a1-4f25-b1a5-7330691ed114 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:12.292003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fbddd-16a1-4f25-b1a5-7330691ed114 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:12.292293 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fbddd-16a1-4f25-b1a5-7330691ed114 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:12.293927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 861/1747] 10.222.0.43 () {54 vars in 912 bytes} [Thu Jun 23 22:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:36:12.295564 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8f84d-2c56-434c-b22a-f5a96cf96b3e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:12.295856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8f84d-2c56-434c-b22a-f5a96cf96b3e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:12.296142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8f84d-2c56-434c-b22a-f5a96cf96b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:12.298107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-75120847-5325-4963-b97d-34bc85776081 admin admin] The token cache is being invalidate because project 19d2156f2b2d455da84804f4343aa70b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:36:12.298107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-75120847-5325-4963-b97d-34bc85776081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:12.300559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-75120847-5325-4963-b97d-34bc85776081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:12.309578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 887/1748] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:36:11 2022] DELETE /identity/v3/projects/19d2156f2b2d455da84804f4343aa70b => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:12.312131 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75ad87-0431-49f0-980a-1270f97a3609 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:12.313085 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75ad87-0431-49f0-980a-1270f97a3609 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:12.313833 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75ad87-0431-49f0-980a-1270f97a3609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:12.367980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-55b8f84d-2c56-434c-b22a-f5a96cf96b3e None None] MFA Rules not processed for user `53c519b28b174ef385dd7e76b85cf504`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:12.374347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-55b8f84d-2c56-434c-b22a-f5a96cf96b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:12.393586 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-6e75ad87-0431-49f0-980a-1270f97a3609 None None] MFA Rules not processed for user `37ba527b96634fc2a48b6ca9597d8191`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:12.397809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-6e75ad87-0431-49f0-980a-1270f97a3609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:12.499579 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:12.526689 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 862/1749] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:12.534071 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8f84d-2c56-434c-b22a-f5a96cf96b3e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:12.535710 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-55b8f84d-2c56-434c-b22a-f5a96cf96b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:12.537888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 888/1750] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 22:36:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:12.545190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75ad87-0431-49f0-980a-1270f97a3609 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:12.546730 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-6e75ad87-0431-49f0-980a-1270f97a3609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:12.653433 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8f84d-2c56-434c-b22a-f5a96cf96b3e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:12.664829 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1179b2f-9eef-4b64-8ccc-cea5a3a685ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:12.672019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1179b2f-9eef-4b64-8ccc-cea5a3a685ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:12.674275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1179b2f-9eef-4b64-8ccc-cea5a3a685ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:12.676786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1179b2f-9eef-4b64-8ccc-cea5a3a685ab admin admin] PATH_INFO: `/v3/users/f2d407e6f5fd4c1b8471508777478d3f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:12.686251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1179b2f-9eef-4b64-8ccc-cea5a3a685ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2d407e6f5fd4c1b8471508777478d3f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:12.689828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1179b2f-9eef-4b64-8ccc-cea5a3a685ab admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:12.692173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:12.702652 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75ad87-0431-49f0-980a-1270f97a3609 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:12.742632 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616018fb-35ce-4ed6-8319-fe71741cc456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:12.743636 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616018fb-35ce-4ed6-8319-fe71741cc456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:12.743885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616018fb-35ce-4ed6-8319-fe71741cc456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:12.747013 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616018fb-35ce-4ed6-8319-fe71741cc456 admin admin] PATH_INFO: `/v3/users/01468d30f7c74480a9eeab7fcaec5889` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:12.749705 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616018fb-35ce-4ed6-8319-fe71741cc456 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01468d30f7c74480a9eeab7fcaec5889)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:12.752991 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616018fb-35ce-4ed6-8319-fe71741cc456 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:12.808206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f1179b2f-9eef-4b64-8ccc-cea5a3a685ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2d407e6f5fd4c1b8471508777478d3f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:12.824260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f1179b2f-9eef-4b64-8ccc-cea5a3a685ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2d407e6f5fd4c1b8471508777478d3f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:12.837611 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f1179b2f-9eef-4b64-8ccc-cea5a3a685ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2d407e6f5fd4c1b8471508777478d3f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:12.839733 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f1179b2f-9eef-4b64-8ccc-cea5a3a685ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2d407e6f5fd4c1b8471508777478d3f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:12.840865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f1179b2f-9eef-4b64-8ccc-cea5a3a685ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2d407e6f5fd4c1b8471508777478d3f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:12.869412 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-616018fb-35ce-4ed6-8319-fe71741cc456 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01468d30f7c74480a9eeab7fcaec5889'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:12.908150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-616018fb-35ce-4ed6-8319-fe71741cc456 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01468d30f7c74480a9eeab7fcaec5889'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:12.913956 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 863/1751] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:36:12 2022] DELETE /identity/v3/users/f2d407e6f5fd4c1b8471508777478d3f => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:12.915019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1179b2f-9eef-4b64-8ccc-cea5a3a685ab admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:12.920469 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f1179b2f-9eef-4b64-8ccc-cea5a3a685ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:12.933877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-616018fb-35ce-4ed6-8319-fe71741cc456 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01468d30f7c74480a9eeab7fcaec5889'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:12.967930 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-616018fb-35ce-4ed6-8319-fe71741cc456 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01468d30f7c74480a9eeab7fcaec5889'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:12.967930 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-616018fb-35ce-4ed6-8319-fe71741cc456 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01468d30f7c74480a9eeab7fcaec5889'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:12.993001 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 889/1752] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:36:12 2022] DELETE /identity/v3/users/01468d30f7c74480a9eeab7fcaec5889 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:12.995470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46a88a-b57d-4319-9bf2-9caf2a4029c7 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:12.995768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46a88a-b57d-4319-9bf2-9caf2a4029c7 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:12.996056 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46a88a-b57d-4319-9bf2-9caf2a4029c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:13.089254 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-1f46a88a-b57d-4319-9bf2-9caf2a4029c7 None None] MFA Rules not processed for user `03486c7630314392b85271455d12aba2`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:13.090729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1f46a88a-b57d-4319-9bf2-9caf2a4029c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:13.152950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1179b2f-9eef-4b64-8ccc-cea5a3a685ab admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:13.290239 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 890/1753] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:13.298866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02426703-5496-4af5-ab31-2944315d1940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:13.299129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88c834-59c3-4883-8e18-ee566187efd7 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:13.299493 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88c834-59c3-4883-8e18-ee566187efd7 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:13.299606 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02426703-5496-4af5-ab31-2944315d1940 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:13.299801 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88c834-59c3-4883-8e18-ee566187efd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:13.300037 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02426703-5496-4af5-ab31-2944315d1940 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:13.300149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02426703-5496-4af5-ab31-2944315d1940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:13.304685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-02426703-5496-4af5-ab31-2944315d1940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:13.332647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02426703-5496-4af5-ab31-2944315d1940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:13.340232 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-3d88c834-59c3-4883-8e18-ee566187efd7 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:13.343515 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02426703-5496-4af5-ab31-2944315d1940 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:13.345043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3d88c834-59c3-4883-8e18-ee566187efd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:13.357931 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:13.365398 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 864/1754] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:13.366496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02426703-5496-4af5-ab31-2944315d1940 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:13.372154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-02426703-5496-4af5-ab31-2944315d1940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:13.512403 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:13.591110 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 891/1755] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:13.598217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331971e-e926-4744-b9d2-51128f2254c4 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:13.600679 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331971e-e926-4744-b9d2-51128f2254c4 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:13.602843 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331971e-e926-4744-b9d2-51128f2254c4 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:13.625283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 892/1756] 10.222.0.43 () {54 vars in 960 bytes} [Thu Jun 23 22:36:13 2022] GET /identity => generated 270 bytes in 33 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:36:13.626263 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331971e-e926-4744-b9d2-51128f2254c4 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:13.636544 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7331971e-e926-4744-b9d2-51128f2254c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:13.670290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02426703-5496-4af5-ab31-2944315d1940 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:13.681573 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331971e-e926-4744-b9d2-51128f2254c4 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:13.692005 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2741318-7d77-47e2-b67e-2c07bf640716 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:13.695695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5d4fc-41cb-4d2e-bad8-65ec67461b52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:13.696882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5d4fc-41cb-4d2e-bad8-65ec67461b52 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:13.698656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5d4fc-41cb-4d2e-bad8-65ec67461b52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:13.698656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5d4fc-41cb-4d2e-bad8-65ec67461b52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:13.709500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2741318-7d77-47e2-b67e-2c07bf640716 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:13.714263 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2741318-7d77-47e2-b67e-2c07bf640716 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:13.717171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2741318-7d77-47e2-b67e-2c07bf640716 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:13.724512 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a2741318-7d77-47e2-b67e-2c07bf640716 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:13.736323 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5d4fc-41cb-4d2e-bad8-65ec67461b52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:13.737816 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5d4fc-41cb-4d2e-bad8-65ec67461b52 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:13.781991 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 893/1757] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:13.787904 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878384ee-5437-44f6-b878-d9690bf8d1b3 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:13.788728 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878384ee-5437-44f6-b878-d9690bf8d1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:13.789422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878384ee-5437-44f6-b878-d9690bf8d1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:13.843578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-878384ee-5437-44f6-b878-d9690bf8d1b3 None None] MFA Rules not processed for user `101864afd59543e699189917391e6ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:13.845895 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-878384ee-5437-44f6-b878-d9690bf8d1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:13.955640 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2741318-7d77-47e2-b67e-2c07bf640716 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:13.961533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2741318-7d77-47e2-b67e-2c07bf640716 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:13.988552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 865/1758] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:14.033997 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2741318-7d77-47e2-b67e-2c07bf640716 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:14.041659 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a2741318-7d77-47e2-b67e-2c07bf640716 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:14.112399 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2741318-7d77-47e2-b67e-2c07bf640716 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:14.141934 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 894/1759] 10.222.0.43 () {58 vars in 1057 bytes} [Thu Jun 23 22:36:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:14.142428 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9b556-057c-4665-acab-758e6bfdb9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:14.144889 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9b556-057c-4665-acab-758e6bfdb9e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:14.145200 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9b556-057c-4665-acab-758e6bfdb9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:14.145494 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9b556-057c-4665-acab-758e6bfdb9e3 admin admin] PATH_INFO: `/v3/projects/707bb2b7fe514daaa9cbf57443e8d58d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:14.150976 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9b556-057c-4665-acab-758e6bfdb9e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=707bb2b7fe514daaa9cbf57443e8d58d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:14.155606 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9b556-057c-4665-acab-758e6bfdb9e3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:14.158125 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878384ee-5437-44f6-b878-d9690bf8d1b3 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:14.176204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878384ee-5437-44f6-b878-d9690bf8d1b3 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:14.191006 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebad6b-e53e-4ad7-b2a1-360f4bee376a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101864afd59543e699189917391e6ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:14.193146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebad6b-e53e-4ad7-b2a1-360f4bee376a service placement] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:14.193614 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebad6b-e53e-4ad7-b2a1-360f4bee376a service placement] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:14.195366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebad6b-e53e-4ad7-b2a1-360f4bee376a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:14.200285 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-14ebad6b-e53e-4ad7-b2a1-360f4bee376a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:14.224811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebad6b-e53e-4ad7-b2a1-360f4bee376a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:14.227636 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebad6b-e53e-4ad7-b2a1-360f4bee376a service placement] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:14.241807 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 895/1760] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:14.319549 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a1c9b556-057c-4665-acab-758e6bfdb9e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '707bb2b7fe514daaa9cbf57443e8d58d', 'request_id': 'req-a1c9b556-057c-4665-acab-758e6bfdb9e3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:14.334670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a1c9b556-057c-4665-acab-758e6bfdb9e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '707bb2b7fe514daaa9cbf57443e8d58d', 'request_id': 'req-a1c9b556-057c-4665-acab-758e6bfdb9e3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:14.363774 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a1c9b556-057c-4665-acab-758e6bfdb9e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '707bb2b7fe514daaa9cbf57443e8d58d', 'request_id': 'req-a1c9b556-057c-4665-acab-758e6bfdb9e3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:14.395864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a1c9b556-057c-4665-acab-758e6bfdb9e3 admin admin] The token cache is being invalidate because project 707bb2b7fe514daaa9cbf57443e8d58d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:36:14.396997 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a1c9b556-057c-4665-acab-758e6bfdb9e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:14.398080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a1c9b556-057c-4665-acab-758e6bfdb9e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:14.404786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 866/1761] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:36:14 2022] DELETE /identity/v3/projects/707bb2b7fe514daaa9cbf57443e8d58d => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:14.410266 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabd2eb-2235-4454-8d82-16455471caa6 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:14.410595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabd2eb-2235-4454-8d82-16455471caa6 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:14.410849 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dabd2eb-2235-4454-8d82-16455471caa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:14.460909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-3dabd2eb-2235-4454-8d82-16455471caa6 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:14.463271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3dabd2eb-2235-4454-8d82-16455471caa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:14.607288 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 896/1762] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:14.615019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b8c5b-de33-40d5-8074-0dd12c3cbfc9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:14.615449 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b8c5b-de33-40d5-8074-0dd12c3cbfc9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:14.615836 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b8c5b-de33-40d5-8074-0dd12c3cbfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:14.650589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3e7b8c5b-de33-40d5-8074-0dd12c3cbfc9 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:14.652116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3e7b8c5b-de33-40d5-8074-0dd12c3cbfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:14.693641 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 867/1763] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:14.694801 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabd2eb-2235-4454-8d82-16455471caa6 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:14.715120 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dabd2eb-2235-4454-8d82-16455471caa6 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:14.751696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e5b14-6f3d-4375-a5f1-765338dea465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:14.752410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e5b14-6f3d-4375-a5f1-765338dea465 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:14.752702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e5b14-6f3d-4375-a5f1-765338dea465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:14.756969 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e5b14-6f3d-4375-a5f1-765338dea465 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:14.757798 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e5b14-6f3d-4375-a5f1-765338dea465 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:36:14.758323 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e5b14-6f3d-4375-a5f1-765338dea465 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:14.762488 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e5b14-6f3d-4375-a5f1-765338dea465 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:14.831292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 897/1764] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:36:14.835848 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b8c5b-de33-40d5-8074-0dd12c3cbfc9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:14.852285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b8c5b-de33-40d5-8074-0dd12c3cbfc9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:14.891713 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bb66d-1b36-42be-8d67-bfa73110376d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:14.891713 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bb66d-1b36-42be-8d67-bfa73110376d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:14.900903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bb66d-1b36-42be-8d67-bfa73110376d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:14.900903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bb66d-1b36-42be-8d67-bfa73110376d admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:14.903944 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bb66d-1b36-42be-8d67-bfa73110376d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:14.915990 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bb66d-1b36-42be-8d67-bfa73110376d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:15.026484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 868/1765] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:14 2022] POST /identity/v3/projects => generated 381 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:15.031070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e5b14-6f3d-4375-a5f1-765338dea465 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:15.045490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e5b14-6f3d-4375-a5f1-765338dea465 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:15.050887 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:15.314914 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbba7ec-4f0f-4ed5-9416-e58e9ba97780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:15.315643 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbba7ec-4f0f-4ed5-9416-e58e9ba97780 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:15.315933 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbba7ec-4f0f-4ed5-9416-e58e9ba97780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:15.316207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbba7ec-4f0f-4ed5-9416-e58e9ba97780 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:15.317085 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbba7ec-4f0f-4ed5-9416-e58e9ba97780 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:15.322524 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbba7ec-4f0f-4ed5-9416-e58e9ba97780 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:15.382258 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 898/1766] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:15 2022] POST /identity/v3/users => generated 291 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:15.387169 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bb66d-1b36-42be-8d67-bfa73110376d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:15.411056 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bb66d-1b36-42be-8d67-bfa73110376d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:15.438354 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407dfad5-ab44-4948-b6df-7b8fb7726385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:15.443329 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407dfad5-ab44-4948-b6df-7b8fb7726385 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:15.444864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407dfad5-ab44-4948-b6df-7b8fb7726385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:15.445154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407dfad5-ab44-4948-b6df-7b8fb7726385 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:15.448084 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407dfad5-ab44-4948-b6df-7b8fb7726385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:15.463495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407dfad5-ab44-4948-b6df-7b8fb7726385 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:15.544758 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 869/1767] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:36:15.548227 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbba7ec-4f0f-4ed5-9416-e58e9ba97780 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:15.570616 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbba7ec-4f0f-4ed5-9416-e58e9ba97780 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:15.590356 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ce7d7-e1f6-4fcb-bce4-cc3c43cca99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:15.594776 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ce7d7-e1f6-4fcb-bce4-cc3c43cca99d admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:15.596933 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ce7d7-e1f6-4fcb-bce4-cc3c43cca99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:15.597270 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ce7d7-e1f6-4fcb-bce4-cc3c43cca99d admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:15.601968 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ce7d7-e1f6-4fcb-bce4-cc3c43cca99d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:15.609600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ce7d7-e1f6-4fcb-bce4-cc3c43cca99d admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:15.641667 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 899/1768] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:15.646617 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407dfad5-ab44-4948-b6df-7b8fb7726385 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:15.665373 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ce7d7-e1f6-4fcb-bce4-cc3c43cca99d admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:15.675944 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-259ce7d7-e1f6-4fcb-bce4-cc3c43cca99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:15.686768 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407dfad5-ab44-4948-b6df-7b8fb7726385 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:15.710355 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d3940-5f08-452d-b4d3-773961dd59ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:15.710964 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d3940-5f08-452d-b4d3-773961dd59ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:15.711213 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d3940-5f08-452d-b4d3-773961dd59ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:15.711489 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d3940-5f08-452d-b4d3-773961dd59ad admin admin] PATH_INFO: `/v3/projects/eaf93f84a3ad4e26a9da4b489cc7f7f1/users/d75f955353c6481798240b17ee2fb8d9/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:15.737033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ce7d7-e1f6-4fcb-bce4-cc3c43cca99d admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:15.756625 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf05cf6-56db-4a7a-9d2c-85632cf82a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:15.758363 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf05cf6-56db-4a7a-9d2c-85632cf82a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:15.759682 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf05cf6-56db-4a7a-9d2c-85632cf82a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:15.759975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf05cf6-56db-4a7a-9d2c-85632cf82a80 admin admin] PATH_INFO: `/v3/projects/d43bc11b97484a04bb667891d05a3411` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:15.764763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf05cf6-56db-4a7a-9d2c-85632cf82a80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d43bc11b97484a04bb667891d05a3411)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:15.766716 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d3940-5f08-452d-b4d3-773961dd59ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf93f84a3ad4e26a9da4b489cc7f7f1, user_id=d75f955353c6481798240b17ee2fb8d9, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:15.769201 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf05cf6-56db-4a7a-9d2c-85632cf82a80 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:15.773294 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d3940-5f08-452d-b4d3-773961dd59ad admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:15.778217 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:15.804730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 870/1769] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:15 2022] PUT /identity/v3/projects/eaf93f84a3ad4e26a9da4b489cc7f7f1/users/d75f955353c6481798240b17ee2fb8d9/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:15.817285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a19e96-981e-4e7b-9e1b-b634a29d3fe6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:15.817624 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a19e96-981e-4e7b-9e1b-b634a29d3fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:15.817856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a19e96-981e-4e7b-9e1b-b634a29d3fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:15.876734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-64a19e96-981e-4e7b-9e1b-b634a29d3fe6 None None] MFA Rules not processed for user `d75f955353c6481798240b17ee2fb8d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:15.877976 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-64a19e96-981e-4e7b-9e1b-b634a29d3fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:15.886514 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7cf05cf6-56db-4a7a-9d2c-85632cf82a80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd43bc11b97484a04bb667891d05a3411', 'request_id': 'req-7cf05cf6-56db-4a7a-9d2c-85632cf82a80'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:15.905747 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7cf05cf6-56db-4a7a-9d2c-85632cf82a80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd43bc11b97484a04bb667891d05a3411', 'request_id': 'req-7cf05cf6-56db-4a7a-9d2c-85632cf82a80'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:15.927553 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7cf05cf6-56db-4a7a-9d2c-85632cf82a80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd43bc11b97484a04bb667891d05a3411', 'request_id': 'req-7cf05cf6-56db-4a7a-9d2c-85632cf82a80'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:15.953636 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7cf05cf6-56db-4a7a-9d2c-85632cf82a80 admin admin] The token cache is being invalidate because project d43bc11b97484a04bb667891d05a3411 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:36:15.954283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7cf05cf6-56db-4a7a-9d2c-85632cf82a80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:15.955226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7cf05cf6-56db-4a7a-9d2c-85632cf82a80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:15.957092 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 900/1770] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:36:15 2022] DELETE /identity/v3/projects/d43bc11b97484a04bb667891d05a3411 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:15.961072 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf05cf6-56db-4a7a-9d2c-85632cf82a80 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:15.962820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7cf05cf6-56db-4a7a-9d2c-85632cf82a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:16.038349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 871/1771] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:36:16.042733 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a19e96-981e-4e7b-9e1b-b634a29d3fe6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:16.046026 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-64a19e96-981e-4e7b-9e1b-b634a29d3fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:16.168324 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a19e96-981e-4e7b-9e1b-b634a29d3fe6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:16.183817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f22cf-6c38-42ba-a3c5-3e4e250a98a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:16.184533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f22cf-6c38-42ba-a3c5-3e4e250a98a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:16.184929 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f22cf-6c38-42ba-a3c5-3e4e250a98a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:16.185320 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f22cf-6c38-42ba-a3c5-3e4e250a98a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:16.186079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f22cf-6c38-42ba-a3c5-3e4e250a98a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:16.188712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f22cf-6c38-42ba-a3c5-3e4e250a98a2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:16.194445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf05cf6-56db-4a7a-9d2c-85632cf82a80 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:16.202664 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4652d-bb5d-4088-9520-b71105bb6ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:16.204022 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4652d-bb5d-4088-9520-b71105bb6ba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:16.204411 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4652d-bb5d-4088-9520-b71105bb6ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:16.204697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4652d-bb5d-4088-9520-b71105bb6ba3 admin admin] PATH_INFO: `/v3/users/51d89d2745a14aac99f5b971816845b3` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:16.209766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb4652d-bb5d-4088-9520-b71105bb6ba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51d89d2745a14aac99f5b971816845b3)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:16.213052 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb4652d-bb5d-4088-9520-b71105bb6ba3 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:16.221984 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 872/1772] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:16 2022] POST /identity/v3/projects => generated 381 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:16.226635 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f22cf-6c38-42ba-a3c5-3e4e250a98a2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:16.235077 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:16.343836 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f22cf-6c38-42ba-a3c5-3e4e250a98a2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:16.352528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-6bb4652d-bb5d-4088-9520-b71105bb6ba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51d89d2745a14aac99f5b971816845b3'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:16.355692 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d3545-8e0a-49e7-87f2-c376559f0b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:16.355939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d3545-8e0a-49e7-87f2-c376559f0b31 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:16.356392 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d3545-8e0a-49e7-87f2-c376559f0b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:16.356699 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d3545-8e0a-49e7-87f2-c376559f0b31 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:16.359518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d3545-8e0a-49e7-87f2-c376559f0b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:16.362736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d3545-8e0a-49e7-87f2-c376559f0b31 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:16.382621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-6bb4652d-bb5d-4088-9520-b71105bb6ba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51d89d2745a14aac99f5b971816845b3'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:16.393029 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:16.395103 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-6bb4652d-bb5d-4088-9520-b71105bb6ba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51d89d2745a14aac99f5b971816845b3'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:16.413251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 873/1773] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:16 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:16.415831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d3545-8e0a-49e7-87f2-c376559f0b31 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:16.420054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-6bb4652d-bb5d-4088-9520-b71105bb6ba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51d89d2745a14aac99f5b971816845b3'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:16.421097 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-6bb4652d-bb5d-4088-9520-b71105bb6ba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51d89d2745a14aac99f5b971816845b3'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:16.437810 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d3545-8e0a-49e7-87f2-c376559f0b31 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:16.438814 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 901/1774] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:36:15 2022] DELETE /identity/v3/users/51d89d2745a14aac99f5b971816845b3 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:16.538543 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c6228-6cca-4bc7-a5e3-d5ed99be6ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:16.539301 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c6228-6cca-4bc7-a5e3-d5ed99be6ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:16.540178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c6228-6cca-4bc7-a5e3-d5ed99be6ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:16.540509 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c6228-6cca-4bc7-a5e3-d5ed99be6ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:16.542784 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c6228-6cca-4bc7-a5e3-d5ed99be6ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:16.545519 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c6228-6cca-4bc7-a5e3-d5ed99be6ef4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:16.587261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 874/1775] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:16.589026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4652d-bb5d-4088-9520-b71105bb6ba3 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:16.591286 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-6bb4652d-bb5d-4088-9520-b71105bb6ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:16.609198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4652d-bb5d-4088-9520-b71105bb6ba3 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:16.615344 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5425d79-44dd-489c-adf9-9bd0361bff9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:16.616833 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5425d79-44dd-489c-adf9-9bd0361bff9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:16.617138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5425d79-44dd-489c-adf9-9bd0361bff9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:16.617716 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5425d79-44dd-489c-adf9-9bd0361bff9a admin admin] PATH_INFO: `/v3/projects/96050575825446f682305eede5b970bf/users/62a83a8375cd486aae2284bc1822f821/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:16.640017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5425d79-44dd-489c-adf9-9bd0361bff9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=96050575825446f682305eede5b970bf, user_id=62a83a8375cd486aae2284bc1822f821, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:16.644631 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5425d79-44dd-489c-adf9-9bd0361bff9a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:16.675335 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 902/1776] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:16 2022] PUT /identity/v3/projects/96050575825446f682305eede5b970bf/users/62a83a8375cd486aae2284bc1822f821/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:16.680719 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c6228-6cca-4bc7-a5e3-d5ed99be6ef4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:16.704863 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c6228-6cca-4bc7-a5e3-d5ed99be6ef4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:16.962649 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7ed1a-39ab-454e-99fd-644d83f02bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:16.963450 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7ed1a-39ab-454e-99fd-644d83f02bdc admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:16.963757 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7ed1a-39ab-454e-99fd-644d83f02bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:16.969049 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7ed1a-39ab-454e-99fd-644d83f02bdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:16.969807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7ed1a-39ab-454e-99fd-644d83f02bdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:16.985783 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7ed1a-39ab-454e-99fd-644d83f02bdc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:17.015206 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5425d79-44dd-489c-adf9-9bd0361bff9a admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:17.017716 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-c5425d79-44dd-489c-adf9-9bd0361bff9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:17.035773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5425d79-44dd-489c-adf9-9bd0361bff9a admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:17.041746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d00f04-7576-4ee1-9074-c6a290fb90d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:17.043013 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d00f04-7576-4ee1-9074-c6a290fb90d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:17.043376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d00f04-7576-4ee1-9074-c6a290fb90d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:17.043726 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d00f04-7576-4ee1-9074-c6a290fb90d3 admin admin] PATH_INFO: `/v3/users/f04108f730bb430b9b8adb6181052e9a` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:17.045203 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 875/1777] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:17.050229 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d00f04-7576-4ee1-9074-c6a290fb90d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f04108f730bb430b9b8adb6181052e9a)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:17.055087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d00f04-7576-4ee1-9074-c6a290fb90d3 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:17.057405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7ed1a-39ab-454e-99fd-644d83f02bdc admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:17.159513 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7ed1a-39ab-454e-99fd-644d83f02bdc admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:17.218240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3af886-eaca-45f6-90a1-26b3bc7c2172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:17.219220 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3af886-eaca-45f6-90a1-26b3bc7c2172 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:17.219508 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3af886-eaca-45f6-90a1-26b3bc7c2172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:17.219792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3af886-eaca-45f6-90a1-26b3bc7c2172 admin admin] PATH_INFO: `/v3/projects/96050575825446f682305eede5b970bf/users/62a83a8375cd486aae2284bc1822f821/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:17.228137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3af886-eaca-45f6-90a1-26b3bc7c2172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96050575825446f682305eede5b970bf, user_id=62a83a8375cd486aae2284bc1822f821, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:17.229393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-94d00f04-7576-4ee1-9074-c6a290fb90d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f04108f730bb430b9b8adb6181052e9a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:17.239132 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3af886-eaca-45f6-90a1-26b3bc7c2172 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:17.287665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 876/1778] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:17 2022] PUT /identity/v3/projects/96050575825446f682305eede5b970bf/users/62a83a8375cd486aae2284bc1822f821/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:17.288007 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5df60-1b9c-427e-b362-a712a88b1623 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:17.288314 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5df60-1b9c-427e-b362-a712a88b1623 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:17.288605 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5df60-1b9c-427e-b362-a712a88b1623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:17.318578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-94d00f04-7576-4ee1-9074-c6a290fb90d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f04108f730bb430b9b8adb6181052e9a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:17.337205 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-17a5df60-1b9c-427e-b362-a712a88b1623 None None] MFA Rules not processed for user `62a83a8375cd486aae2284bc1822f821`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:17.339257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-17a5df60-1b9c-427e-b362-a712a88b1623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:17.343754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-94d00f04-7576-4ee1-9074-c6a290fb90d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f04108f730bb430b9b8adb6181052e9a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:17.359378 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-94d00f04-7576-4ee1-9074-c6a290fb90d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f04108f730bb430b9b8adb6181052e9a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:17.362866 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-94d00f04-7576-4ee1-9074-c6a290fb90d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f04108f730bb430b9b8adb6181052e9a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:17.385977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 903/1779] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:36:17 2022] DELETE /identity/v3/users/f04108f730bb430b9b8adb6181052e9a => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:17.394346 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b27e4-b62b-45be-b85c-09bf160f46ec None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:17.394644 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b27e4-b62b-45be-b85c-09bf160f46ec None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:17.395000 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b27e4-b62b-45be-b85c-09bf160f46ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:17.525605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-0b4b27e4-b62b-45be-b85c-09bf160f46ec None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:17.529178 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-0b4b27e4-b62b-45be-b85c-09bf160f46ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:17.619512 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 877/1780] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:36:17.621959 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d0850-218b-4e94-bd2c-3cf6c4461035 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:17.623054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d0850-218b-4e94-bd2c-3cf6c4461035 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:17.623505 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d0850-218b-4e94-bd2c-3cf6c4461035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:17.636839 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:17.650456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:DELETE=20,keystone:SELECT=77,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:17.678366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a39d0850-218b-4e94-bd2c-3cf6c4461035 None None] MFA Rules not processed for user `dc246e144afc46698000f7f53ef6489e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:17.679629 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a39d0850-218b-4e94-bd2c-3cf6c4461035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:17.724457 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 904/1781] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:17.726489 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b27e4-b62b-45be-b85c-09bf160f46ec None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:17.739535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-0b4b27e4-b62b-45be-b85c-09bf160f46ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:17.853412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 878/1782] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:17.854593 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d0850-218b-4e94-bd2c-3cf6c4461035 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:17.860067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a39d0850-218b-4e94-bd2c-3cf6c4461035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:17.972845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b27e4-b62b-45be-b85c-09bf160f46ec None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:17.990252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550afb1a-5529-43d9-af3f-1aa2dcaa6b55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:17.991377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550afb1a-5529-43d9-af3f-1aa2dcaa6b55 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:17.999158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550afb1a-5529-43d9-af3f-1aa2dcaa6b55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:17.999641 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550afb1a-5529-43d9-af3f-1aa2dcaa6b55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:18.006363 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-550afb1a-5529-43d9-af3f-1aa2dcaa6b55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:18.118047 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d0850-218b-4e94-bd2c-3cf6c4461035 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:18.144475 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a5b1c-1da1-41eb-adde-d9f8b7082783 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:18.147343 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a5b1c-1da1-41eb-adde-d9f8b7082783 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:18.148188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a5b1c-1da1-41eb-adde-d9f8b7082783 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:18.149964 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a5b1c-1da1-41eb-adde-d9f8b7082783 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:18.242048 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550afb1a-5529-43d9-af3f-1aa2dcaa6b55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:18.244374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550afb1a-5529-43d9-af3f-1aa2dcaa6b55 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:18.275352 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 905/1783] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:18.277765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550afb1a-5529-43d9-af3f-1aa2dcaa6b55 service cinder] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:18.284551 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-550afb1a-5529-43d9-af3f-1aa2dcaa6b55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:18.453176 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a5b1c-1da1-41eb-adde-d9f8b7082783 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:18.454652 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a5b1c-1da1-41eb-adde-d9f8b7082783 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:18.456397 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550afb1a-5529-43d9-af3f-1aa2dcaa6b55 service cinder] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:18.474405 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1d94c-95f8-4a1b-842d-148dffeb92ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:18.475264 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1d94c-95f8-4a1b-842d-148dffeb92ce service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:18.475698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1d94c-95f8-4a1b-842d-148dffeb92ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:18.476411 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1d94c-95f8-4a1b-842d-148dffeb92ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:18.520038 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 879/1784] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:18.522133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1d94c-95f8-4a1b-842d-148dffeb92ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:18.523228 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a5b1c-1da1-41eb-adde-d9f8b7082783 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:18.526322 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1d94c-95f8-4a1b-842d-148dffeb92ce service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:18.527092 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c43a5b1c-1da1-41eb-adde-d9f8b7082783 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:18.571451 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 906/1785] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:18.613316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a5b1c-1da1-41eb-adde-d9f8b7082783 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:18.636541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9a67f-39ee-4d74-aea3-2f40f5f08880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:18.641250 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9a67f-39ee-4d74-aea3-2f40f5f08880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:18.644510 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9a67f-39ee-4d74-aea3-2f40f5f08880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:18.644994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9a67f-39ee-4d74-aea3-2f40f5f08880 admin admin] PATH_INFO: `/v3/projects/3617c63403f349bb884c08d9a0fb0cf5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:18.649862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9a67f-39ee-4d74-aea3-2f40f5f08880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3617c63403f349bb884c08d9a0fb0cf5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:18.661425 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9a67f-39ee-4d74-aea3-2f40f5f08880 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:18.822039 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1d94c-95f8-4a1b-842d-148dffeb92ce service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:18.841238 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b0d9a67f-39ee-4d74-aea3-2f40f5f08880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3617c63403f349bb884c08d9a0fb0cf5', 'request_id': 'req-b0d9a67f-39ee-4d74-aea3-2f40f5f08880'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:18.844502 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1d94c-95f8-4a1b-842d-148dffeb92ce service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:18.851764 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:18.865931 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b0d9a67f-39ee-4d74-aea3-2f40f5f08880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3617c63403f349bb884c08d9a0fb0cf5', 'request_id': 'req-b0d9a67f-39ee-4d74-aea3-2f40f5f08880'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:18.884456 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b0d9a67f-39ee-4d74-aea3-2f40f5f08880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3617c63403f349bb884c08d9a0fb0cf5', 'request_id': 'req-b0d9a67f-39ee-4d74-aea3-2f40f5f08880'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:18.923448 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b0d9a67f-39ee-4d74-aea3-2f40f5f08880 admin admin] The token cache is being invalidate because project 3617c63403f349bb884c08d9a0fb0cf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:36:18.937288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b0d9a67f-39ee-4d74-aea3-2f40f5f08880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:18.940403 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:18.942208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b0d9a67f-39ee-4d74-aea3-2f40f5f08880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:18.948752 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 880/1786] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:36:18 2022] DELETE /identity/v3/projects/3617c63403f349bb884c08d9a0fb0cf5 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:18.951875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfa815-ad71-4b0c-8d95-56608207dc5a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:18.952245 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfa815-ad71-4b0c-8d95-56608207dc5a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:18.952415 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbfa815-ad71-4b0c-8d95-56608207dc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:19.002364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-acbfa815-ad71-4b0c-8d95-56608207dc5a None None] MFA Rules not processed for user `df1aeb2be41c49819b811ecfa47d4155`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:19.006112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-acbfa815-ad71-4b0c-8d95-56608207dc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:19.009533 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4a45b-b8a2-4a6a-bab6-df390a3708a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:19.011010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4a45b-b8a2-4a6a-bab6-df390a3708a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:19.011301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4a45b-b8a2-4a6a-bab6-df390a3708a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:19.011576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4a45b-b8a2-4a6a-bab6-df390a3708a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:19.019732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-93c4a45b-b8a2-4a6a-bab6-df390a3708a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:19.180917 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4a45b-b8a2-4a6a-bab6-df390a3708a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:19.184218 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4a45b-b8a2-4a6a-bab6-df390a3708a3 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:19.223388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 907/1787] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:36:19.225323 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ae01f-ff7a-4758-a42c-a0d7d3008bf2 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:19.228111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ae01f-ff7a-4758-a42c-a0d7d3008bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:19.229030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ae01f-ff7a-4758-a42c-a0d7d3008bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:19.282053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 881/1788] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 22:36:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:19.307777 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-892ae01f-ff7a-4758-a42c-a0d7d3008bf2 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:19.309282 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-892ae01f-ff7a-4758-a42c-a0d7d3008bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:19.566768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 908/1789] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:19.576464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10706b0e-e652-425b-8c0d-ce43ff7e44a2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:19.576943 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10706b0e-e652-425b-8c0d-ce43ff7e44a2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:19.577265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10706b0e-e652-425b-8c0d-ce43ff7e44a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:19.620237 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-10706b0e-e652-425b-8c0d-ce43ff7e44a2 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:19.634716 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-10706b0e-e652-425b-8c0d-ce43ff7e44a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:19.643946 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ae01f-ff7a-4758-a42c-a0d7d3008bf2 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:19.645409 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-892ae01f-ff7a-4758-a42c-a0d7d3008bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:19.674140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 882/1790] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:19.678808 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10706b0e-e652-425b-8c0d-ce43ff7e44a2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:19.698081 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10706b0e-e652-425b-8c0d-ce43ff7e44a2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:19.722340 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15310251-8977-43c3-9db6-36b76fdfdc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:19.723060 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15310251-8977-43c3-9db6-36b76fdfdc12 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:19.723431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15310251-8977-43c3-9db6-36b76fdfdc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:19.723720 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15310251-8977-43c3-9db6-36b76fdfdc12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:19.724563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15310251-8977-43c3-9db6-36b76fdfdc12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:36:19.724912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15310251-8977-43c3-9db6-36b76fdfdc12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:19.726742 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15310251-8977-43c3-9db6-36b76fdfdc12 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:19.769613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 883/1791] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:36:19.779761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15310251-8977-43c3-9db6-36b76fdfdc12 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:19.823843 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15310251-8977-43c3-9db6-36b76fdfdc12 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:19.846629 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabc33b-3f7d-4e2b-a371-348b29d44e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:19.849029 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabc33b-3f7d-4e2b-a371-348b29d44e5e admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:19.851637 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabc33b-3f7d-4e2b-a371-348b29d44e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:19.852183 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabc33b-3f7d-4e2b-a371-348b29d44e5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:19.853124 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabc33b-3f7d-4e2b-a371-348b29d44e5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:19.861711 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabc33b-3f7d-4e2b-a371-348b29d44e5e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:19.871392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ae01f-ff7a-4758-a42c-a0d7d3008bf2 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:19.885686 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468aa02-65be-49a6-985d-babc9d87efd3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ce35c43e8294f4cbc880f2219a6eb76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:19.887084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468aa02-65be-49a6-985d-babc9d87efd3 service swift] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:19.887084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468aa02-65be-49a6-985d-babc9d87efd3 service swift] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:19.887084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468aa02-65be-49a6-985d-babc9d87efd3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:19.904857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 884/1792] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:19 2022] POST /identity/v3/projects => generated 405 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:19.907268 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabc33b-3f7d-4e2b-a371-348b29d44e5e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:19.922323 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468aa02-65be-49a6-985d-babc9d87efd3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:19.924071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468aa02-65be-49a6-985d-babc9d87efd3 service swift] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:19.951302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 909/1793] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 22:36:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 22:36:20.217017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabc33b-3f7d-4e2b-a371-348b29d44e5e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:20.230381 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae29c-e8c9-4320-9082-50fe525f9115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:20.231690 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae29c-e8c9-4320-9082-50fe525f9115 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:20.232035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae29c-e8c9-4320-9082-50fe525f9115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:20.232326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae29c-e8c9-4320-9082-50fe525f9115 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:20.233042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fae29c-e8c9-4320-9082-50fe525f9115 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:20.241795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fae29c-e8c9-4320-9082-50fe525f9115 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:20.315409 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468aa02-65be-49a6-985d-babc9d87efd3 service swift] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:20.325047 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 885/1794] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:19 2022] POST /identity/v3/users => generated 303 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:20.341942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468aa02-65be-49a6-985d-babc9d87efd3 service swift] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:20.359137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f14f1-a6a2-4ac1-811d-c608ae1e72ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:20.359896 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f14f1-a6a2-4ac1-811d-c608ae1e72ed admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:20.360195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f14f1-a6a2-4ac1-811d-c608ae1e72ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:20.360486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f14f1-a6a2-4ac1-811d-c608ae1e72ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:20.361285 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f14f1-a6a2-4ac1-811d-c608ae1e72ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:20.369287 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f14f1-a6a2-4ac1-811d-c608ae1e72ed admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:20.455603 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae29c-e8c9-4320-9082-50fe525f9115 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:20.463019 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 910/1795] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:20.473411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae29c-e8c9-4320-9082-50fe525f9115 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:20.498997 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e9e30-689f-463d-86f1-8f1d3f4bbaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:20.499713 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e9e30-689f-463d-86f1-8f1d3f4bbaac admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:20.499996 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e9e30-689f-463d-86f1-8f1d3f4bbaac admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:20.500275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e9e30-689f-463d-86f1-8f1d3f4bbaac admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:20.501022 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e9e30-689f-463d-86f1-8f1d3f4bbaac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:20.502659 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e9e30-689f-463d-86f1-8f1d3f4bbaac admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:20.533998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 886/1796] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:36:20.542032 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f14f1-a6a2-4ac1-811d-c608ae1e72ed admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:20.588970 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f14f1-a6a2-4ac1-811d-c608ae1e72ed admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:20.610742 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd017653-c823-4af9-b1e8-569e911065c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:20.611933 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd017653-c823-4af9-b1e8-569e911065c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:20.616485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd017653-c823-4af9-b1e8-569e911065c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:20.616864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd017653-c823-4af9-b1e8-569e911065c3 admin admin] PATH_INFO: `/v3/projects/1f2a574c6e764b53bf8144badf2ab027/users/13f6decd0e80403e95cc609900b7396b/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:20.631268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:20.686986 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd017653-c823-4af9-b1e8-569e911065c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f2a574c6e764b53bf8144badf2ab027, user_id=13f6decd0e80403e95cc609900b7396b, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:20.697737 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd017653-c823-4af9-b1e8-569e911065c3 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:20.747988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 911/1797] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:20 2022] PUT /identity/v3/projects/1f2a574c6e764b53bf8144badf2ab027/users/13f6decd0e80403e95cc609900b7396b/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:20.749483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c068bb-ee59-4938-a92d-bf16797db5af None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:20.749985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c068bb-ee59-4938-a92d-bf16797db5af None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:20.750078 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c068bb-ee59-4938-a92d-bf16797db5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:20.813617 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-73c068bb-ee59-4938-a92d-bf16797db5af None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:20.814744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-73c068bb-ee59-4938-a92d-bf16797db5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:20.874223 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:21.043587 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 887/1798] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:21.217007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd017653-c823-4af9-b1e8-569e911065c3 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:21.221169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-fd017653-c823-4af9-b1e8-569e911065c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:21.227882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c068bb-ee59-4938-a92d-bf16797db5af None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:21.229552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-73c068bb-ee59-4938-a92d-bf16797db5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:21.269379 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c068bb-ee59-4938-a92d-bf16797db5af None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:21.289784 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1c2d5-3ead-431e-bc3d-fa1fe78790e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:21.290728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1c2d5-3ead-431e-bc3d-fa1fe78790e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:21.296936 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1c2d5-3ead-431e-bc3d-fa1fe78790e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:21.297332 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1c2d5-3ead-431e-bc3d-fa1fe78790e3 admin admin] PATH_INFO: `/v3/projects/192b9fd67e49457290d6e89b8d0a2fb6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:21.302148 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1c2d5-3ead-431e-bc3d-fa1fe78790e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=192b9fd67e49457290d6e89b8d0a2fb6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:21.304369 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1c2d5-3ead-431e-bc3d-fa1fe78790e3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:21.452336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd017653-c823-4af9-b1e8-569e911065c3 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:21.464485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e98f3-522c-47d4-ad38-6f6dff88c18c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:21.465434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e98f3-522c-47d4-ad38-6f6dff88c18c service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:21.480693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e98f3-522c-47d4-ad38-6f6dff88c18c service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:21.482542 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e98f3-522c-47d4-ad38-6f6dff88c18c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:21.518954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-abf1c2d5-3ead-431e-bc3d-fa1fe78790e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '192b9fd67e49457290d6e89b8d0a2fb6', 'request_id': 'req-abf1c2d5-3ead-431e-bc3d-fa1fe78790e3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:21.524187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e98f3-522c-47d4-ad38-6f6dff88c18c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:21.527554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e98f3-522c-47d4-ad38-6f6dff88c18c service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:21.543821 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-abf1c2d5-3ead-431e-bc3d-fa1fe78790e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '192b9fd67e49457290d6e89b8d0a2fb6', 'request_id': 'req-abf1c2d5-3ead-431e-bc3d-fa1fe78790e3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:21.557588 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 912/1799] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:21.562964 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-abf1c2d5-3ead-431e-bc3d-fa1fe78790e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '192b9fd67e49457290d6e89b8d0a2fb6', 'request_id': 'req-abf1c2d5-3ead-431e-bc3d-fa1fe78790e3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:21.596915 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-abf1c2d5-3ead-431e-bc3d-fa1fe78790e3 admin admin] The token cache is being invalidate because project 192b9fd67e49457290d6e89b8d0a2fb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:36:21.601289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-abf1c2d5-3ead-431e-bc3d-fa1fe78790e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:21.604786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-abf1c2d5-3ead-431e-bc3d-fa1fe78790e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:21.608643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 888/1800] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:36:21 2022] DELETE /identity/v3/projects/192b9fd67e49457290d6e89b8d0a2fb6 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:36:21.612965 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e98f3-522c-47d4-ad38-6f6dff88c18c service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:21.628929 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e98f3-522c-47d4-ad38-6f6dff88c18c service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:21.762392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c81c18-3818-4a52-9e7c-ba0edb8c411f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:21.763652 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c81c18-3818-4a52-9e7c-ba0edb8c411f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:21.764083 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c81c18-3818-4a52-9e7c-ba0edb8c411f admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:21.769522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c81c18-3818-4a52-9e7c-ba0edb8c411f admin admin] PATH_INFO: `/v3/users/baefb455c62c4844a56cc8f98d16287b` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:21.769522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c81c18-3818-4a52-9e7c-ba0edb8c411f admin admin] RBAC: Authorizing `identity:delete_user(user_id=baefb455c62c4844a56cc8f98d16287b)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:21.776347 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c81c18-3818-4a52-9e7c-ba0edb8c411f admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:21.924226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d1c81c18-3818-4a52-9e7c-ba0edb8c411f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baefb455c62c4844a56cc8f98d16287b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:21.960973 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d1c81c18-3818-4a52-9e7c-ba0edb8c411f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baefb455c62c4844a56cc8f98d16287b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:21.975420 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d1c81c18-3818-4a52-9e7c-ba0edb8c411f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baefb455c62c4844a56cc8f98d16287b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:22.015251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d1c81c18-3818-4a52-9e7c-ba0edb8c411f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baefb455c62c4844a56cc8f98d16287b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:22.028940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d1c81c18-3818-4a52-9e7c-ba0edb8c411f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baefb455c62c4844a56cc8f98d16287b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:22.036613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5dfc7-cbfd-4399-b1d9-57529bf69990 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:22.037708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5dfc7-cbfd-4399-b1d9-57529bf69990 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:22.038254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5dfc7-cbfd-4399-b1d9-57529bf69990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:22.045679 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 913/1801] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:36:21 2022] DELETE /identity/v3/users/baefb455c62c4844a56cc8f98d16287b => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:22.113497 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d9f5dfc7-cbfd-4399-b1d9-57529bf69990 None None] MFA Rules not processed for user `37ba527b96634fc2a48b6ca9597d8191`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:22.115545 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d9f5dfc7-cbfd-4399-b1d9-57529bf69990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:22.251513 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 889/1802] 10.222.0.55 () {58 vars in 1015 bytes} [Thu Jun 23 22:36:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:36:22.369681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c81c18-3818-4a52-9e7c-ba0edb8c411f admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:22.371989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d1c81c18-3818-4a52-9e7c-ba0edb8c411f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:22.382279 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:22.418691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5dfc7-cbfd-4399-b1d9-57529bf69990 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:22.420890 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d9f5dfc7-cbfd-4399-b1d9-57529bf69990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:22.453350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5dfc7-cbfd-4399-b1d9-57529bf69990 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:22.465597 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9c203-a7e6-4145-bb1a-69c3ec292254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:22.473238 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9c203-a7e6-4145-bb1a-69c3ec292254 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:22.474591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9c203-a7e6-4145-bb1a-69c3ec292254 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:22.475101 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9c203-a7e6-4145-bb1a-69c3ec292254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:22.514589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9c203-a7e6-4145-bb1a-69c3ec292254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:22.516404 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9c203-a7e6-4145-bb1a-69c3ec292254 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:22.548066 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 890/1803] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:36:22.567827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c81c18-3818-4a52-9e7c-ba0edb8c411f admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:22.573994 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101e8ca-81a3-41a6-be3b-74d859eef3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:22.577245 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101e8ca-81a3-41a6-be3b-74d859eef3f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:22.577762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101e8ca-81a3-41a6-be3b-74d859eef3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:22.578267 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101e8ca-81a3-41a6-be3b-74d859eef3f7 admin admin] PATH_INFO: `/v3/users/418fb2e877884959a2c027e8f742167f` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:22.586014 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101e8ca-81a3-41a6-be3b-74d859eef3f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=418fb2e877884959a2c027e8f742167f)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:22.588354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101e8ca-81a3-41a6-be3b-74d859eef3f7 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:22.671675 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-9101e8ca-81a3-41a6-be3b-74d859eef3f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '418fb2e877884959a2c027e8f742167f'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:22.694470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-9101e8ca-81a3-41a6-be3b-74d859eef3f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '418fb2e877884959a2c027e8f742167f'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:22.709464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-9101e8ca-81a3-41a6-be3b-74d859eef3f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '418fb2e877884959a2c027e8f742167f'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:22.725354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-9101e8ca-81a3-41a6-be3b-74d859eef3f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '418fb2e877884959a2c027e8f742167f'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:22.725696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-9101e8ca-81a3-41a6-be3b-74d859eef3f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '418fb2e877884959a2c027e8f742167f'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:22.737373 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 914/1804] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:36:22 2022] DELETE /identity/v3/users/418fb2e877884959a2c027e8f742167f => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:22.741528 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27256169-2829-4c64-8ea6-c37862ed0b0f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:22.742116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27256169-2829-4c64-8ea6-c37862ed0b0f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:22.744107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27256169-2829-4c64-8ea6-c37862ed0b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:22.795992 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-27256169-2829-4c64-8ea6-c37862ed0b0f None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:22.800198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-27256169-2829-4c64-8ea6-c37862ed0b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:22.825031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101e8ca-81a3-41a6-be3b-74d859eef3f7 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:22.826543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-9101e8ca-81a3-41a6-be3b-74d859eef3f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:22.872803 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:22.922797 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 891/1805] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:36:23.038769 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27256169-2829-4c64-8ea6-c37862ed0b0f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:23.041532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-27256169-2829-4c64-8ea6-c37862ed0b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:23.053535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101e8ca-81a3-41a6-be3b-74d859eef3f7 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:23.069231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad6c59-a3dd-4cbd-a2cf-61f7be901a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:23.070829 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad6c59-a3dd-4cbd-a2cf-61f7be901a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:23.071161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad6c59-a3dd-4cbd-a2cf-61f7be901a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:23.071531 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad6c59-a3dd-4cbd-a2cf-61f7be901a1e admin admin] PATH_INFO: `/v3/users/d75f955353c6481798240b17ee2fb8d9` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:23.073446 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad6c59-a3dd-4cbd-a2cf-61f7be901a1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d75f955353c6481798240b17ee2fb8d9)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:23.083719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad6c59-a3dd-4cbd-a2cf-61f7be901a1e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:23.152821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:23.194432 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d0ad6c59-a3dd-4cbd-a2cf-61f7be901a1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd75f955353c6481798240b17ee2fb8d9'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:23.234615 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d0ad6c59-a3dd-4cbd-a2cf-61f7be901a1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd75f955353c6481798240b17ee2fb8d9'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:23.248662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d0ad6c59-a3dd-4cbd-a2cf-61f7be901a1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd75f955353c6481798240b17ee2fb8d9'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:23.288364 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d0ad6c59-a3dd-4cbd-a2cf-61f7be901a1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd75f955353c6481798240b17ee2fb8d9'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:23.289681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d0ad6c59-a3dd-4cbd-a2cf-61f7be901a1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd75f955353c6481798240b17ee2fb8d9'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:23.298959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 915/1806] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:36:22 2022] DELETE /identity/v3/users/d75f955353c6481798240b17ee2fb8d9 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:23.303101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad6c59-a3dd-4cbd-a2cf-61f7be901a1e admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:23.308026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d0ad6c59-a3dd-4cbd-a2cf-61f7be901a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:23.319728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27256169-2829-4c64-8ea6-c37862ed0b0f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:23.460094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99a9549-ab24-444f-b658-26ff6610ea5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:23.474995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a9549-ab24-444f-b658-26ff6610ea5a service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:23.475356 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a9549-ab24-444f-b658-26ff6610ea5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:23.475696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a9549-ab24-444f-b658-26ff6610ea5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:23.478857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f99a9549-ab24-444f-b658-26ff6610ea5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:23.496238 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a9549-ab24-444f-b658-26ff6610ea5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:23.500427 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99a9549-ab24-444f-b658-26ff6610ea5a service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:23.513532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad6c59-a3dd-4cbd-a2cf-61f7be901a1e admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:23.519085 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 892/1807] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:23.521451 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b6e5a-4f9b-48cb-a8fd-d84f4cff4c03 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:23.521797 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b6e5a-4f9b-48cb-a8fd-d84f4cff4c03 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:23.522027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b6e5a-4f9b-48cb-a8fd-d84f4cff4c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:23.556974 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-433b6e5a-4f9b-48cb-a8fd-d84f4cff4c03 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:23.557333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ad6fb-bd71-4695-9928-63a3bdcdda15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:23.558509 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-433b6e5a-4f9b-48cb-a8fd-d84f4cff4c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:23.565286 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ad6fb-bd71-4695-9928-63a3bdcdda15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:23.572830 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ad6fb-bd71-4695-9928-63a3bdcdda15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:23.573330 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ad6fb-bd71-4695-9928-63a3bdcdda15 admin admin] PATH_INFO: `/v3/projects/9d44ba037d414cb2b0471762e7d83141` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:23.576830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 893/1808] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:23.581134 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ad6fb-bd71-4695-9928-63a3bdcdda15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d44ba037d414cb2b0471762e7d83141)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:23.592222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ad6fb-bd71-4695-9928-63a3bdcdda15 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:23.612934 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b6e5a-4f9b-48cb-a8fd-d84f4cff4c03 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:23.615499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-433b6e5a-4f9b-48cb-a8fd-d84f4cff4c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:23.646283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b6e5a-4f9b-48cb-a8fd-d84f4cff4c03 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:23.657899 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d6e5d-b565-4555-bdd5-035457438363 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:23.660359 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d6e5d-b565-4555-bdd5-035457438363 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:23.661952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d6e5d-b565-4555-bdd5-035457438363 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:23.662361 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d6e5d-b565-4555-bdd5-035457438363 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:23.692966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d6e5d-b565-4555-bdd5-035457438363 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:23.702380 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d6e5d-b565-4555-bdd5-035457438363 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:23.760455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 894/1809] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:23.764707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-244ad6fb-bd71-4695-9928-63a3bdcdda15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d44ba037d414cb2b0471762e7d83141', 'request_id': 'req-244ad6fb-bd71-4695-9928-63a3bdcdda15'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:23.824780 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-244ad6fb-bd71-4695-9928-63a3bdcdda15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d44ba037d414cb2b0471762e7d83141', 'request_id': 'req-244ad6fb-bd71-4695-9928-63a3bdcdda15'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:23.850877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-244ad6fb-bd71-4695-9928-63a3bdcdda15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d44ba037d414cb2b0471762e7d83141', 'request_id': 'req-244ad6fb-bd71-4695-9928-63a3bdcdda15'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:23.911422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-244ad6fb-bd71-4695-9928-63a3bdcdda15 admin admin] The token cache is being invalidate because project 9d44ba037d414cb2b0471762e7d83141 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:36:23.913737 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-244ad6fb-bd71-4695-9928-63a3bdcdda15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:23.914949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-244ad6fb-bd71-4695-9928-63a3bdcdda15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:23.918208 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 916/1810] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:36:23 2022] DELETE /identity/v3/projects/9d44ba037d414cb2b0471762e7d83141 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:23.924019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998264a-e329-4641-b569-21010033e217 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:23.924431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998264a-e329-4641-b569-21010033e217 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:23.924822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998264a-e329-4641-b569-21010033e217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:23.962074 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e998264a-e329-4641-b569-21010033e217 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:23.967166 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e998264a-e329-4641-b569-21010033e217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:24.185476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a6c99-8b47-4b80-b28a-1ccc0bee6ab1 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:24.185706 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 895/1811] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:24.186578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a6c99-8b47-4b80-b28a-1ccc0bee6ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:24.187053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a6c99-8b47-4b80-b28a-1ccc0bee6ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:24.220925 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-e34a6c99-8b47-4b80-b28a-1ccc0bee6ab1 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:24.226783 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e34a6c99-8b47-4b80-b28a-1ccc0bee6ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:24.263634 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 917/1812] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:36:24.267848 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998264a-e329-4641-b569-21010033e217 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:24.287013 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998264a-e329-4641-b569-21010033e217 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:24.330729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1933ef6-3961-4780-919f-8c6e5ccca6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:24.332180 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1933ef6-3961-4780-919f-8c6e5ccca6ea admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:24.337572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1933ef6-3961-4780-919f-8c6e5ccca6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:24.339127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1933ef6-3961-4780-919f-8c6e5ccca6ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:24.342394 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1933ef6-3961-4780-919f-8c6e5ccca6ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:36:24.345534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1933ef6-3961-4780-919f-8c6e5ccca6ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:24.351474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1933ef6-3961-4780-919f-8c6e5ccca6ea admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:24.376604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 896/1813] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:36:24.377854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a6c99-8b47-4b80-b28a-1ccc0bee6ab1 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:24.387244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a6c99-8b47-4b80-b28a-1ccc0bee6ab1 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:24.431947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339a857-25cd-46af-b0cf-b53fc521bd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:24.432647 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339a857-25cd-46af-b0cf-b53fc521bd14 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:24.432949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339a857-25cd-46af-b0cf-b53fc521bd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:24.433221 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339a857-25cd-46af-b0cf-b53fc521bd14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:24.433933 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339a857-25cd-46af-b0cf-b53fc521bd14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:24.435985 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339a857-25cd-46af-b0cf-b53fc521bd14 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:24.539979 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 918/1814] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:24 2022] POST /identity/v3/projects => generated 383 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:24.545992 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1933ef6-3961-4780-919f-8c6e5ccca6ea admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:24.563697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1933ef6-3961-4780-919f-8c6e5ccca6ea admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:24.578639 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:24.765182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299da25c-7a5d-4b24-b5e4-ef90d98b44fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:24.766140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299da25c-7a5d-4b24-b5e4-ef90d98b44fa admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:24.766589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299da25c-7a5d-4b24-b5e4-ef90d98b44fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:24.767051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299da25c-7a5d-4b24-b5e4-ef90d98b44fa admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:24.768063 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299da25c-7a5d-4b24-b5e4-ef90d98b44fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:24.777179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299da25c-7a5d-4b24-b5e4-ef90d98b44fa admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:24.835223 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 897/1815] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:24 2022] POST /identity/v3/users => generated 292 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:24.836397 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339a857-25cd-46af-b0cf-b53fc521bd14 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:24.843762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:24.853971 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339a857-25cd-46af-b0cf-b53fc521bd14 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:24.880142 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf3e2c-7b94-4948-aee1-377471c31d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:24.880821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf3e2c-7b94-4948-aee1-377471c31d04 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:24.881109 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf3e2c-7b94-4948-aee1-377471c31d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:24.881393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf3e2c-7b94-4948-aee1-377471c31d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:24.882131 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf3e2c-7b94-4948-aee1-377471c31d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:24.895882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf3e2c-7b94-4948-aee1-377471c31d04 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:24.918382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299da25c-7a5d-4b24-b5e4-ef90d98b44fa admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:24.923388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 919/1816] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:24.941933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299da25c-7a5d-4b24-b5e4-ef90d98b44fa admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:24.969254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04adc1ec-c40b-413b-a7d6-8a32aa98beb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:24.971507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04adc1ec-c40b-413b-a7d6-8a32aa98beb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:24.973876 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04adc1ec-c40b-413b-a7d6-8a32aa98beb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:24.974690 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04adc1ec-c40b-413b-a7d6-8a32aa98beb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:24.977856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04adc1ec-c40b-413b-a7d6-8a32aa98beb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:24.980158 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04adc1ec-c40b-413b-a7d6-8a32aa98beb1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:25.035191 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf3e2c-7b94-4948-aee1-377471c31d04 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:25.037914 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 898/1817] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:25.080802 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf3e2c-7b94-4948-aee1-377471c31d04 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:25.099314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76170657-2d22-4350-b112-fa4f248485a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:25.100316 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76170657-2d22-4350-b112-fa4f248485a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:25.100755 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76170657-2d22-4350-b112-fa4f248485a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:25.101107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76170657-2d22-4350-b112-fa4f248485a2 admin admin] PATH_INFO: `/v3/projects/3d69a6f283a4406cb4a522a44849363f/users/21fe10e2f309460d8cb01f9cd3fbf450/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:25.148977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76170657-2d22-4350-b112-fa4f248485a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d69a6f283a4406cb4a522a44849363f, user_id=21fe10e2f309460d8cb01f9cd3fbf450, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:25.153185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76170657-2d22-4350-b112-fa4f248485a2 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:25.222629 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 920/1818] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:25 2022] PUT /identity/v3/projects/3d69a6f283a4406cb4a522a44849363f/users/21fe10e2f309460d8cb01f9cd3fbf450/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:25.225236 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a12ad1-edc8-430f-9eec-d419f3d8a857 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:25.226154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a12ad1-edc8-430f-9eec-d419f3d8a857 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:25.226598 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a12ad1-edc8-430f-9eec-d419f3d8a857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:25.318331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a9a12ad1-edc8-430f-9eec-d419f3d8a857 None None] MFA Rules not processed for user `21fe10e2f309460d8cb01f9cd3fbf450`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:25.320038 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a9a12ad1-edc8-430f-9eec-d419f3d8a857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:25.633574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 899/1819] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:25.657416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76170657-2d22-4350-b112-fa4f248485a2 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:25.660052 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-76170657-2d22-4350-b112-fa4f248485a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:25.812974 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76170657-2d22-4350-b112-fa4f248485a2 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:25.819723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019d185-e9f6-4d8f-96dc-c81fdc47cdac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:25.820765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019d185-e9f6-4d8f-96dc-c81fdc47cdac service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:25.821208 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019d185-e9f6-4d8f-96dc-c81fdc47cdac service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:25.821632 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019d185-e9f6-4d8f-96dc-c81fdc47cdac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:25.832853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0019d185-e9f6-4d8f-96dc-c81fdc47cdac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:25.834732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0019d185-e9f6-4d8f-96dc-c81fdc47cdac service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:25.847396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 921/1820] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:26.350789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c65098-00e7-4f70-888d-415d96a704d4 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:26.351413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c65098-00e7-4f70-888d-415d96a704d4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:26.351413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c65098-00e7-4f70-888d-415d96a704d4 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:26.353070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 900/1821] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 22:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:36:26.484415 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18984f7-978f-4087-b918-439ce4162f24 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:26.484849 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18984f7-978f-4087-b918-439ce4162f24 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:26.485144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18984f7-978f-4087-b918-439ce4162f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:26.526547 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-f18984f7-978f-4087-b918-439ce4162f24 None None] MFA Rules not processed for user `35d686923ef541819de3c77847d38511`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:26.527834 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-f18984f7-978f-4087-b918-439ce4162f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:26.542711 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c65098-00e7-4f70-888d-415d96a704d4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:26.545754 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e1c65098-00e7-4f70-888d-415d96a704d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:26.728373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c65098-00e7-4f70-888d-415d96a704d4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:26.734679 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff06d52-e687-41c3-8829-65ec60daca58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:26.734679 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff06d52-e687-41c3-8829-65ec60daca58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:26.735162 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff06d52-e687-41c3-8829-65ec60daca58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:26.735423 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff06d52-e687-41c3-8829-65ec60daca58 admin admin] PATH_INFO: `/v3/projects/c6a8c615fcea43f79b97d91853b4ac3e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:26.737516 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff06d52-e687-41c3-8829-65ec60daca58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a8c615fcea43f79b97d91853b4ac3e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:26.738882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 922/1822] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 22:36:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:26.740598 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff06d52-e687-41c3-8829-65ec60daca58 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:26.796395 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:26.883320 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-dff06d52-e687-41c3-8829-65ec60daca58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a8c615fcea43f79b97d91853b4ac3e', 'request_id': 'req-dff06d52-e687-41c3-8829-65ec60daca58'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:26.895898 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-dff06d52-e687-41c3-8829-65ec60daca58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a8c615fcea43f79b97d91853b4ac3e', 'request_id': 'req-dff06d52-e687-41c3-8829-65ec60daca58'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:26.918459 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-dff06d52-e687-41c3-8829-65ec60daca58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a8c615fcea43f79b97d91853b4ac3e', 'request_id': 'req-dff06d52-e687-41c3-8829-65ec60daca58'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:26.962873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-dff06d52-e687-41c3-8829-65ec60daca58 admin admin] The token cache is being invalidate because project c6a8c615fcea43f79b97d91853b4ac3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:36:26.963231 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-dff06d52-e687-41c3-8829-65ec60daca58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:26.963970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-dff06d52-e687-41c3-8829-65ec60daca58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:26.966011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 901/1823] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:36:26 2022] DELETE /identity/v3/projects/c6a8c615fcea43f79b97d91853b4ac3e => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:26.969914 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18984f7-978f-4087-b918-439ce4162f24 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:26.992042 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18984f7-978f-4087-b918-439ce4162f24 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:27.140529 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad807d22-3ad6-4402-8e8d-e7f80e914baa None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:27.140833 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad807d22-3ad6-4402-8e8d-e7f80e914baa None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:27.141127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad807d22-3ad6-4402-8e8d-e7f80e914baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:27.163495 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80f9c6-d311-4b56-b055-310879121782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:27.164675 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80f9c6-d311-4b56-b055-310879121782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:27.165157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80f9c6-d311-4b56-b055-310879121782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:27.167134 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80f9c6-d311-4b56-b055-310879121782 admin admin] PATH_INFO: `/v3/users/a7afffcffe034d2aad7ec6c00c4dc6ef` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:27.171434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80f9c6-d311-4b56-b055-310879121782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7afffcffe034d2aad7ec6c00c4dc6ef)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:27.175838 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80f9c6-d311-4b56-b055-310879121782 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:27.190353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:27.227952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ad807d22-3ad6-4402-8e8d-e7f80e914baa None None] MFA Rules not processed for user `13f6decd0e80403e95cc609900b7396b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:27.234712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ad807d22-3ad6-4402-8e8d-e7f80e914baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:27.303823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0f80f9c6-d311-4b56-b055-310879121782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7afffcffe034d2aad7ec6c00c4dc6ef'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:27.336604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0f80f9c6-d311-4b56-b055-310879121782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7afffcffe034d2aad7ec6c00c4dc6ef'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:27.347191 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0f80f9c6-d311-4b56-b055-310879121782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7afffcffe034d2aad7ec6c00c4dc6ef'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:27.357529 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0f80f9c6-d311-4b56-b055-310879121782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7afffcffe034d2aad7ec6c00c4dc6ef'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:27.358305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0f80f9c6-d311-4b56-b055-310879121782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7afffcffe034d2aad7ec6c00c4dc6ef'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:27.378168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 923/1824] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:36:26 2022] DELETE /identity/v3/users/a7afffcffe034d2aad7ec6c00c4dc6ef => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:27.378168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80f9c6-d311-4b56-b055-310879121782 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:27.396575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-0f80f9c6-d311-4b56-b055-310879121782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:27.437380 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 902/1825] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:27 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:27.439801 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad807d22-3ad6-4402-8e8d-e7f80e914baa None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:27.448517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ad807d22-3ad6-4402-8e8d-e7f80e914baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:27.515820 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad807d22-3ad6-4402-8e8d-e7f80e914baa None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:27.574497 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80f9c6-d311-4b56-b055-310879121782 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:27.594138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d45003-438d-437f-808a-8d54c28e8395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:27.594138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d45003-438d-437f-808a-8d54c28e8395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:27.594662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d45003-438d-437f-808a-8d54c28e8395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:27.594963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d45003-438d-437f-808a-8d54c28e8395 admin admin] PATH_INFO: `/v3/projects/eaf93f84a3ad4e26a9da4b489cc7f7f1` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:27.599747 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d45003-438d-437f-808a-8d54c28e8395 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaf93f84a3ad4e26a9da4b489cc7f7f1)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:27.602281 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d45003-438d-437f-808a-8d54c28e8395 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:27.731185 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609344c-595d-4ce2-b6ab-b2c3f9d64351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:27.746786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609344c-595d-4ce2-b6ab-b2c3f9d64351 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:27.748593 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609344c-595d-4ce2-b6ab-b2c3f9d64351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:27.750381 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609344c-595d-4ce2-b6ab-b2c3f9d64351 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:27.751237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-87d45003-438d-437f-808a-8d54c28e8395 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaf93f84a3ad4e26a9da4b489cc7f7f1', 'request_id': 'req-87d45003-438d-437f-808a-8d54c28e8395'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:27.753657 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609344c-595d-4ce2-b6ab-b2c3f9d64351 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:27.759226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609344c-595d-4ce2-b6ab-b2c3f9d64351 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:27.771900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-87d45003-438d-437f-808a-8d54c28e8395 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaf93f84a3ad4e26a9da4b489cc7f7f1', 'request_id': 'req-87d45003-438d-437f-808a-8d54c28e8395'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:27.795656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-87d45003-438d-437f-808a-8d54c28e8395 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaf93f84a3ad4e26a9da4b489cc7f7f1', 'request_id': 'req-87d45003-438d-437f-808a-8d54c28e8395'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:27.827872 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-87d45003-438d-437f-808a-8d54c28e8395 admin admin] The token cache is being invalidate because project eaf93f84a3ad4e26a9da4b489cc7f7f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:36:27.830441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-87d45003-438d-437f-808a-8d54c28e8395 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:27.831618 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-87d45003-438d-437f-808a-8d54c28e8395 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:27.835547 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 924/1826] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:36:27 2022] DELETE /identity/v3/projects/eaf93f84a3ad4e26a9da4b489cc7f7f1 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:27.835915 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 903/1827] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:27 2022] POST /identity/v3/projects => generated 405 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:27.840871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d45003-438d-437f-808a-8d54c28e8395 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:27.841285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609344c-595d-4ce2-b6ab-b2c3f9d64351 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:27.843744 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-87d45003-438d-437f-808a-8d54c28e8395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:27.845766 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1609344c-595d-4ce2-b6ab-b2c3f9d64351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:27.886442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:27.936383 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:28.089740 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d45003-438d-437f-808a-8d54c28e8395 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:28.090328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609344c-595d-4ce2-b6ab-b2c3f9d64351 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:28.112686 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbe9fc-fd3c-49db-8e5c-f06df7b55adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:28.113972 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60ecbc-739b-40d9-9ab2-87b71445f162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:28.114802 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60ecbc-739b-40d9-9ab2-87b71445f162 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:28.116628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbe9fc-fd3c-49db-8e5c-f06df7b55adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:28.117773 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60ecbc-739b-40d9-9ab2-87b71445f162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:28.118107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60ecbc-739b-40d9-9ab2-87b71445f162 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:28.118837 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60ecbc-739b-40d9-9ab2-87b71445f162 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:28.120029 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbe9fc-fd3c-49db-8e5c-f06df7b55adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:28.120360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbe9fc-fd3c-49db-8e5c-f06df7b55adc admin admin] PATH_INFO: `/v3/users/62a83a8375cd486aae2284bc1822f821` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:28.125759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbe9fc-fd3c-49db-8e5c-f06df7b55adc admin admin] RBAC: Authorizing `identity:delete_user(user_id=62a83a8375cd486aae2284bc1822f821)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:28.130849 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60ecbc-739b-40d9-9ab2-87b71445f162 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:28.140087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bbe9fc-fd3c-49db-8e5c-f06df7b55adc admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:28.214705 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 904/1828] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:27 2022] POST /identity/v3/users => generated 309 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:28.217972 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60ecbc-739b-40d9-9ab2-87b71445f162 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:28.272336 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60ecbc-739b-40d9-9ab2-87b71445f162 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:28.290035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6d3dc-5156-4cf2-a95b-2c114f0d565e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:28.290730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6d3dc-5156-4cf2-a95b-2c114f0d565e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:28.291038 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6d3dc-5156-4cf2-a95b-2c114f0d565e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:28.298080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6d3dc-5156-4cf2-a95b-2c114f0d565e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:28.303745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6d3dc-5156-4cf2-a95b-2c114f0d565e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:28.306343 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6d3dc-5156-4cf2-a95b-2c114f0d565e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:28.347262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-88bbe9fc-fd3c-49db-8e5c-f06df7b55adc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62a83a8375cd486aae2284bc1822f821'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:28.363948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-88bbe9fc-fd3c-49db-8e5c-f06df7b55adc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62a83a8375cd486aae2284bc1822f821'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:28.373291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-88bbe9fc-fd3c-49db-8e5c-f06df7b55adc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62a83a8375cd486aae2284bc1822f821'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:28.374724 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 905/1829] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:28.377769 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6d3dc-5156-4cf2-a95b-2c114f0d565e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:28.387739 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-48f6d3dc-5156-4cf2-a95b-2c114f0d565e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:28.407044 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-88bbe9fc-fd3c-49db-8e5c-f06df7b55adc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62a83a8375cd486aae2284bc1822f821'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:28.408694 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-88bbe9fc-fd3c-49db-8e5c-f06df7b55adc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62a83a8375cd486aae2284bc1822f821'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:28.418125 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 925/1830] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:36:27 2022] DELETE /identity/v3/users/62a83a8375cd486aae2284bc1822f821 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:28.422201 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbe9fc-fd3c-49db-8e5c-f06df7b55adc admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:28.424996 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-88bbe9fc-fd3c-49db-8e5c-f06df7b55adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:28.596682 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6d3dc-5156-4cf2-a95b-2c114f0d565e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:28.600548 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbe9fc-fd3c-49db-8e5c-f06df7b55adc admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:28.628998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670accc-9e3d-49a6-9547-2ec97a662861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:28.629626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670accc-9e3d-49a6-9547-2ec97a662861 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:28.629925 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670accc-9e3d-49a6-9547-2ec97a662861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:28.633575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670accc-9e3d-49a6-9547-2ec97a662861 admin admin] PATH_INFO: `/v3/projects/61456fd5f2cc44c8aa7ba760ae870ce9/users/23c20a474a1341acb8a1b2c71a93addf/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:28.677546 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670accc-9e3d-49a6-9547-2ec97a662861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61456fd5f2cc44c8aa7ba760ae870ce9, user_id=23c20a474a1341acb8a1b2c71a93addf, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:28.686457 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670accc-9e3d-49a6-9547-2ec97a662861 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:28.717247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 926/1831] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:28 2022] PUT /identity/v3/projects/61456fd5f2cc44c8aa7ba760ae870ce9/users/23c20a474a1341acb8a1b2c71a93addf/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:28.721421 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670accc-9e3d-49a6-9547-2ec97a662861 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:28.757196 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978018b-86a5-4c0e-b9b5-02c64a777deb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:28.758145 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978018b-86a5-4c0e-b9b5-02c64a777deb service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:28.758344 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978018b-86a5-4c0e-b9b5-02c64a777deb service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:28.758827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:28.760056 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978018b-86a5-4c0e-b9b5-02c64a777deb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:28.765036 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1978018b-86a5-4c0e-b9b5-02c64a777deb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:28.782643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978018b-86a5-4c0e-b9b5-02c64a777deb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:28.785976 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978018b-86a5-4c0e-b9b5-02c64a777deb service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:28.815256 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 906/1832] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:28.869358 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670accc-9e3d-49a6-9547-2ec97a662861 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:28.877880 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d5157-16ca-4b10-b909-a43cba57594c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:28.878749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d5157-16ca-4b10-b909-a43cba57594c admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:28.878870 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d5157-16ca-4b10-b909-a43cba57594c admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:28.879437 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d5157-16ca-4b10-b909-a43cba57594c admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:28.880168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d5157-16ca-4b10-b909-a43cba57594c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:28.887677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d5157-16ca-4b10-b909-a43cba57594c admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:28.920617 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 927/1833] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:36:28.922258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978018b-86a5-4c0e-b9b5-02c64a777deb service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:28.946105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978018b-86a5-4c0e-b9b5-02c64a777deb service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:28.974672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f89696-6e7b-4bfc-b94e-d19e1add1cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:28.975822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f89696-6e7b-4bfc-b94e-d19e1add1cb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:28.976289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f89696-6e7b-4bfc-b94e-d19e1add1cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:28.976715 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f89696-6e7b-4bfc-b94e-d19e1add1cb0 admin admin] PATH_INFO: `/v3/projects/61456fd5f2cc44c8aa7ba760ae870ce9/users/23c20a474a1341acb8a1b2c71a93addf/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:28.979728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f89696-6e7b-4bfc-b94e-d19e1add1cb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61456fd5f2cc44c8aa7ba760ae870ce9, user_id=23c20a474a1341acb8a1b2c71a93addf, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:28.984188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f89696-6e7b-4bfc-b94e-d19e1add1cb0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:29.025085 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 907/1834] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:28 2022] PUT /identity/v3/projects/61456fd5f2cc44c8aa7ba760ae870ce9/users/23c20a474a1341acb8a1b2c71a93addf/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:29.541079 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d5157-16ca-4b10-b909-a43cba57594c admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:29.543709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-355d5157-16ca-4b10-b909-a43cba57594c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:29.569324 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdd300-96ff-43ea-8223-98d7323488df None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:29.569955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdd300-96ff-43ea-8223-98d7323488df None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:29.570545 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdd300-96ff-43ea-8223-98d7323488df None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:29.573642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 908/1835] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 22:36:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:36:29.593127 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d5157-16ca-4b10-b909-a43cba57594c admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:29.619048 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1d99b-9792-4285-952c-454b1cd7aa77 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:29.619560 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1d99b-9792-4285-952c-454b1cd7aa77 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:29.619636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1d99b-9792-4285-952c-454b1cd7aa77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:29.680114 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-41e1d99b-9792-4285-952c-454b1cd7aa77 None None] MFA Rules not processed for user `35d686923ef541819de3c77847d38511`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:29.681433 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-41e1d99b-9792-4285-952c-454b1cd7aa77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:29.689555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:29.720869 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477cb6f-fd65-4491-b087-5f6513e73c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:29.722397 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477cb6f-fd65-4491-b087-5f6513e73c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:29.722691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477cb6f-fd65-4491-b087-5f6513e73c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:29.723003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477cb6f-fd65-4491-b087-5f6513e73c16 admin admin] PATH_INFO: `/v3/projects/96050575825446f682305eede5b970bf` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:29.725410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477cb6f-fd65-4491-b087-5f6513e73c16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96050575825446f682305eede5b970bf)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:29.728995 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477cb6f-fd65-4491-b087-5f6513e73c16 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:29.776629 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 909/1836] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 22:36:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:29.798689 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-3477cb6f-fd65-4491-b087-5f6513e73c16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96050575825446f682305eede5b970bf', 'request_id': 'req-3477cb6f-fd65-4491-b087-5f6513e73c16'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:29.807933 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-3477cb6f-fd65-4491-b087-5f6513e73c16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96050575825446f682305eede5b970bf', 'request_id': 'req-3477cb6f-fd65-4491-b087-5f6513e73c16'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:29.824077 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-3477cb6f-fd65-4491-b087-5f6513e73c16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96050575825446f682305eede5b970bf', 'request_id': 'req-3477cb6f-fd65-4491-b087-5f6513e73c16'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:29.837307 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-3477cb6f-fd65-4491-b087-5f6513e73c16 admin admin] The token cache is being invalidate because project 96050575825446f682305eede5b970bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:36:29.837610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-3477cb6f-fd65-4491-b087-5f6513e73c16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:29.838319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-3477cb6f-fd65-4491-b087-5f6513e73c16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:29.839922 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 928/1837] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:36:29 2022] DELETE /identity/v3/projects/96050575825446f682305eede5b970bf => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:29.844968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0f5f8-5465-4153-97dc-a9e2771b2854 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:29.845270 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0f5f8-5465-4153-97dc-a9e2771b2854 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:29.845559 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0f5f8-5465-4153-97dc-a9e2771b2854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:29.874287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1ee0f5f8-5465-4153-97dc-a9e2771b2854 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:29.875530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1ee0f5f8-5465-4153-97dc-a9e2771b2854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:29.944212 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 910/1838] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:29.953505 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d4812-eb2f-4737-be23-62f264f646ed None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:29.953901 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d4812-eb2f-4737-be23-62f264f646ed None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:29.954407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d4812-eb2f-4737-be23-62f264f646ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:29.979232 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-dd5d4812-eb2f-4737-be23-62f264f646ed None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:29.980494 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-dd5d4812-eb2f-4737-be23-62f264f646ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:29.989448 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 929/1839] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:29.994035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0f5f8-5465-4153-97dc-a9e2771b2854 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:30.004837 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0f5f8-5465-4153-97dc-a9e2771b2854 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:30.009931 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ee7dd-f56b-489d-a04d-9a6b56763396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:30.010611 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ee7dd-f56b-489d-a04d-9a6b56763396 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:30.010897 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ee7dd-f56b-489d-a04d-9a6b56763396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:30.011229 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ee7dd-f56b-489d-a04d-9a6b56763396 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:30.011969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ee7dd-f56b-489d-a04d-9a6b56763396 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:36:30.012281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ee7dd-f56b-489d-a04d-9a6b56763396 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:30.014120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ee7dd-f56b-489d-a04d-9a6b56763396 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:30.026775 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 911/1840] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:36:30.033181 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d4812-eb2f-4737-be23-62f264f646ed None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:30.045109 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d4812-eb2f-4737-be23-62f264f646ed None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:30.050355 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f932fd9-d7db-417a-b95b-bae642a8cf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:30.051083 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f932fd9-d7db-417a-b95b-bae642a8cf9a admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:30.051401 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f932fd9-d7db-417a-b95b-bae642a8cf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:30.051768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f932fd9-d7db-417a-b95b-bae642a8cf9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:30.052514 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f932fd9-d7db-417a-b95b-bae642a8cf9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:30.054449 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f932fd9-d7db-417a-b95b-bae642a8cf9a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:30.083885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 930/1841] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:30 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:30.088141 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ee7dd-f56b-489d-a04d-9a6b56763396 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:30.099204 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ee7dd-f56b-489d-a04d-9a6b56763396 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:30.161225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3624732-ef07-4216-aa53-2c6e07f4a133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:30.162056 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3624732-ef07-4216-aa53-2c6e07f4a133 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:30.162491 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3624732-ef07-4216-aa53-2c6e07f4a133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:30.162838 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3624732-ef07-4216-aa53-2c6e07f4a133 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:30.163776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3624732-ef07-4216-aa53-2c6e07f4a133 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:30.165765 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3624732-ef07-4216-aa53-2c6e07f4a133 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:30.190482 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 912/1842] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:30 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:30.192116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f932fd9-d7db-417a-b95b-bae642a8cf9a admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:30.203464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f932fd9-d7db-417a-b95b-bae642a8cf9a admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:30.208383 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75344f93-9378-4f18-8b9f-3e4a89728827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:30.209063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75344f93-9378-4f18-8b9f-3e4a89728827 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:30.209354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75344f93-9378-4f18-8b9f-3e4a89728827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:30.210276 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75344f93-9378-4f18-8b9f-3e4a89728827 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:30.210613 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75344f93-9378-4f18-8b9f-3e4a89728827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:30.212331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75344f93-9378-4f18-8b9f-3e4a89728827 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:30.224694 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 931/1843] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:30.228849 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3624732-ef07-4216-aa53-2c6e07f4a133 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:30.239534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3624732-ef07-4216-aa53-2c6e07f4a133 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:30.244281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b4a00-18a1-4fab-8d0c-43d1235bc597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:30.244954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b4a00-18a1-4fab-8d0c-43d1235bc597 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:30.245257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b4a00-18a1-4fab-8d0c-43d1235bc597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:30.245531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b4a00-18a1-4fab-8d0c-43d1235bc597 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:30.246562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9b4a00-18a1-4fab-8d0c-43d1235bc597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:30.248494 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9b4a00-18a1-4fab-8d0c-43d1235bc597 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:30.259047 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 913/1844] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:36:30.263211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75344f93-9378-4f18-8b9f-3e4a89728827 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:30.269531 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:30.274802 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75344f93-9378-4f18-8b9f-3e4a89728827 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:30.287043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed50695-9390-43de-a33d-c7b1d125a8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:30.287955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed50695-9390-43de-a33d-c7b1d125a8c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:30.288238 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed50695-9390-43de-a33d-c7b1d125a8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:30.288514 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed50695-9390-43de-a33d-c7b1d125a8c3 admin admin] PATH_INFO: `/v3/projects/68146ddf17d146a29ad9a23581e0ca6e/users/06245c8e10d947af87088187487762f6/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:30.305578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed50695-9390-43de-a33d-c7b1d125a8c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68146ddf17d146a29ad9a23581e0ca6e, user_id=06245c8e10d947af87088187487762f6, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:30.308263 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed50695-9390-43de-a33d-c7b1d125a8c3 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:30.324530 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 932/1845] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:30 2022] PUT /identity/v3/projects/68146ddf17d146a29ad9a23581e0ca6e/users/06245c8e10d947af87088187487762f6/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:30.330158 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7d6ed-eb77-4c93-b98c-dba61d536f95 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:30.330566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7d6ed-eb77-4c93-b98c-dba61d536f95 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:30.330946 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7d6ed-eb77-4c93-b98c-dba61d536f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:30.348602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ccf7d6ed-eb77-4c93-b98c-dba61d536f95 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:30.349836 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ccf7d6ed-eb77-4c93-b98c-dba61d536f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:30.354192 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:30.436857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 914/1846] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:36:30.447439 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed50695-9390-43de-a33d-c7b1d125a8c3 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:30.449187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-aed50695-9390-43de-a33d-c7b1d125a8c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:30.469269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7d6ed-eb77-4c93-b98c-dba61d536f95 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:30.470790 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ccf7d6ed-eb77-4c93-b98c-dba61d536f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:30.485715 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7d6ed-eb77-4c93-b98c-dba61d536f95 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:30.501855 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dc148-bf41-449e-83ea-6cd955a6a58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:30.502759 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dc148-bf41-449e-83ea-6cd955a6a58b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:30.503482 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dc148-bf41-449e-83ea-6cd955a6a58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:30.503782 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dc148-bf41-449e-83ea-6cd955a6a58b admin admin] PATH_INFO: `/v3/projects/131d836799eb4701acd5ac2eaac0b85e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:30.505026 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39dc148-bf41-449e-83ea-6cd955a6a58b admin admin] RBAC: Authorizing `identity:delete_project(project_id=131d836799eb4701acd5ac2eaac0b85e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:30.507998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39dc148-bf41-449e-83ea-6cd955a6a58b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:30.547808 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed50695-9390-43de-a33d-c7b1d125a8c3 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:30.562598 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76809e9b-fa8d-4ab6-9827-28237b429f92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:30.563594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76809e9b-fa8d-4ab6-9827-28237b429f92 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:30.564217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76809e9b-fa8d-4ab6-9827-28237b429f92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:30.564606 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76809e9b-fa8d-4ab6-9827-28237b429f92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:30.577491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76809e9b-fa8d-4ab6-9827-28237b429f92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:30.579309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76809e9b-fa8d-4ab6-9827-28237b429f92 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:30.584383 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f39dc148-bf41-449e-83ea-6cd955a6a58b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '131d836799eb4701acd5ac2eaac0b85e', 'request_id': 'req-f39dc148-bf41-449e-83ea-6cd955a6a58b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:30.591754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 933/1847] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:30.595271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f39dc148-bf41-449e-83ea-6cd955a6a58b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '131d836799eb4701acd5ac2eaac0b85e', 'request_id': 'req-f39dc148-bf41-449e-83ea-6cd955a6a58b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:30.598429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76809e9b-fa8d-4ab6-9827-28237b429f92 service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:30.600102 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-76809e9b-fa8d-4ab6-9827-28237b429f92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:30.602966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f39dc148-bf41-449e-83ea-6cd955a6a58b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '131d836799eb4701acd5ac2eaac0b85e', 'request_id': 'req-f39dc148-bf41-449e-83ea-6cd955a6a58b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:30.618198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f39dc148-bf41-449e-83ea-6cd955a6a58b admin admin] The token cache is being invalidate because project 131d836799eb4701acd5ac2eaac0b85e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:36:30.618499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f39dc148-bf41-449e-83ea-6cd955a6a58b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:30.619228 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f39dc148-bf41-449e-83ea-6cd955a6a58b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:30.621609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 915/1848] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:36:30 2022] DELETE /identity/v3/projects/131d836799eb4701acd5ac2eaac0b85e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:30.625914 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dc148-bf41-449e-83ea-6cd955a6a58b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:30.628158 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f39dc148-bf41-449e-83ea-6cd955a6a58b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:30.724786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dc148-bf41-449e-83ea-6cd955a6a58b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:30.730448 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059733a-1129-40cf-8116-152c88f8e1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:30.731549 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059733a-1129-40cf-8116-152c88f8e1a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:30.731939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059733a-1129-40cf-8116-152c88f8e1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:30.732319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059733a-1129-40cf-8116-152c88f8e1a0 admin admin] PATH_INFO: `/v3/users/8d06a77800884a87969186ae5c21f938` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:30.733749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059733a-1129-40cf-8116-152c88f8e1a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d06a77800884a87969186ae5c21f938)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:30.735846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059733a-1129-40cf-8116-152c88f8e1a0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:30.745369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76809e9b-fa8d-4ab6-9827-28237b429f92 service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:30.806455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:30.821124 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6059733a-1129-40cf-8116-152c88f8e1a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d06a77800884a87969186ae5c21f938'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:30.843714 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6059733a-1129-40cf-8116-152c88f8e1a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d06a77800884a87969186ae5c21f938'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:30.853566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6059733a-1129-40cf-8116-152c88f8e1a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d06a77800884a87969186ae5c21f938'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:30.855377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6059733a-1129-40cf-8116-152c88f8e1a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d06a77800884a87969186ae5c21f938'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:30.856556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6059733a-1129-40cf-8116-152c88f8e1a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d06a77800884a87969186ae5c21f938'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:30.866526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff840c7-761c-40f2-91d7-f448ba390659 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:30.867167 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff840c7-761c-40f2-91d7-f448ba390659 service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:30.869710 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff840c7-761c-40f2-91d7-f448ba390659 service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:30.870063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff840c7-761c-40f2-91d7-f448ba390659 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:30.879452 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-eff840c7-761c-40f2-91d7-f448ba390659 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:30.898762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff840c7-761c-40f2-91d7-f448ba390659 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:30.900867 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff840c7-761c-40f2-91d7-f448ba390659 service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:30.901326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 916/1849] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:36:30 2022] DELETE /identity/v3/users/8d06a77800884a87969186ae5c21f938 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:36:30.929702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 934/1850] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:31.244891 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3474d4e-ec94-4342-a793-f04afaa9037f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:31.250209 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3474d4e-ec94-4342-a793-f04afaa9037f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:31.250679 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3474d4e-ec94-4342-a793-f04afaa9037f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:31.315285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f3474d4e-ec94-4342-a793-f04afaa9037f None None] MFA Rules not processed for user `23c20a474a1341acb8a1b2c71a93addf`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:31.317084 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f3474d4e-ec94-4342-a793-f04afaa9037f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:31.485695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 917/1851] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:31 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:31.981761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff840c7-761c-40f2-91d7-f448ba390659 service nova] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:31.983452 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-eff840c7-761c-40f2-91d7-f448ba390659 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:31.999253 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:32.013871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff840c7-761c-40f2-91d7-f448ba390659 service nova] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:32.118798 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690e800-4753-4905-b4c6-dae2bb59e3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:32.127253 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690e800-4753-4905-b4c6-dae2bb59e3d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:32.127253 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690e800-4753-4905-b4c6-dae2bb59e3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:32.127253 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690e800-4753-4905-b4c6-dae2bb59e3d6 admin admin] PATH_INFO: `/v3/projects/178847ac4e104fe98412714873a387f3` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:32.134859 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690e800-4753-4905-b4c6-dae2bb59e3d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=178847ac4e104fe98412714873a387f3)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:32.137247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690e800-4753-4905-b4c6-dae2bb59e3d6 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:32.227402 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acac2a5-8399-493e-a48d-d2771523e839 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:32.227992 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acac2a5-8399-493e-a48d-d2771523e839 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:32.228165 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acac2a5-8399-493e-a48d-d2771523e839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:32.257588 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b690e800-4753-4905-b4c6-dae2bb59e3d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '178847ac4e104fe98412714873a387f3', 'request_id': 'req-b690e800-4753-4905-b4c6-dae2bb59e3d6'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:32.270393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b690e800-4753-4905-b4c6-dae2bb59e3d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '178847ac4e104fe98412714873a387f3', 'request_id': 'req-b690e800-4753-4905-b4c6-dae2bb59e3d6'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:32.298096 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b690e800-4753-4905-b4c6-dae2bb59e3d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '178847ac4e104fe98412714873a387f3', 'request_id': 'req-b690e800-4753-4905-b4c6-dae2bb59e3d6'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:32.304271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2acac2a5-8399-493e-a48d-d2771523e839 None None] MFA Rules not processed for user `06245c8e10d947af87088187487762f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:32.306604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2acac2a5-8399-493e-a48d-d2771523e839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:32.334766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b690e800-4753-4905-b4c6-dae2bb59e3d6 admin admin] The token cache is being invalidate because project 178847ac4e104fe98412714873a387f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:36:32.335688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b690e800-4753-4905-b4c6-dae2bb59e3d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:32.337189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b690e800-4753-4905-b4c6-dae2bb59e3d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:32.346718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 935/1852] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:36:31 2022] DELETE /identity/v3/projects/178847ac4e104fe98412714873a387f3 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:32.348528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d10fb1-3db2-4894-9128-fbc028a56855 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:32.353890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d10fb1-3db2-4894-9128-fbc028a56855 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:32.354058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d10fb1-3db2-4894-9128-fbc028a56855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:32.392740 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-05d10fb1-3db2-4894-9128-fbc028a56855 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:32.395642 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-05d10fb1-3db2-4894-9128-fbc028a56855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:32.478217 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 918/1853] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:32 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:32.482090 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acac2a5-8399-493e-a48d-d2771523e839 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:32.488945 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2acac2a5-8399-493e-a48d-d2771523e839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:32.589755 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 936/1854] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:32.592675 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbc496-dd23-4183-8427-19c800a02b8d None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:32.593049 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbc496-dd23-4183-8427-19c800a02b8d None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:32.593440 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbc496-dd23-4183-8427-19c800a02b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:32.622305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:32.636988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-42cbc496-dd23-4183-8427-19c800a02b8d None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:32.639614 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-42cbc496-dd23-4183-8427-19c800a02b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:32.674617 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 937/1855] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:32.675623 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cbc496-dd23-4183-8427-19c800a02b8d None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:32.695620 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cbc496-dd23-4183-8427-19c800a02b8d None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:32.709464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370075c-946c-47f6-ba57-69f86b712c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:32.709798 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370075c-946c-47f6-ba57-69f86b712c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:32.710090 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370075c-946c-47f6-ba57-69f86b712c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:32.710376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370075c-946c-47f6-ba57-69f86b712c20 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:32.711154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370075c-946c-47f6-ba57-69f86b712c20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:36:32.711564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370075c-946c-47f6-ba57-69f86b712c20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:32.713079 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370075c-946c-47f6-ba57-69f86b712c20 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:32.740883 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acac2a5-8399-493e-a48d-d2771523e839 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:32.752550 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 938/1856] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:36:32.753533 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370075c-946c-47f6-ba57-69f86b712c20 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:32.761857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891ccc6-55b3-47cf-aede-b26ce96a34f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:32.764371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891ccc6-55b3-47cf-aede-b26ce96a34f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:32.764769 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891ccc6-55b3-47cf-aede-b26ce96a34f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:32.765422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891ccc6-55b3-47cf-aede-b26ce96a34f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:32.766564 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891ccc6-55b3-47cf-aede-b26ce96a34f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:32.770267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891ccc6-55b3-47cf-aede-b26ce96a34f6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:32.786989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370075c-946c-47f6-ba57-69f86b712c20 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:32.796030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56122d3-9f53-43ac-8eed-e0ae5bf436cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:32.797180 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56122d3-9f53-43ac-8eed-e0ae5bf436cf admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:32.797745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56122d3-9f53-43ac-8eed-e0ae5bf436cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:32.798130 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56122d3-9f53-43ac-8eed-e0ae5bf436cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:32.799302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56122d3-9f53-43ac-8eed-e0ae5bf436cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:32.802679 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56122d3-9f53-43ac-8eed-e0ae5bf436cf admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:32.827476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:32.859655 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 939/1857] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:32 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:32.865655 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56122d3-9f53-43ac-8eed-e0ae5bf436cf admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:32.954686 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 919/1858] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:32 2022] POST /identity/v3/projects => generated 371 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:32.959818 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891ccc6-55b3-47cf-aede-b26ce96a34f6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:33.036454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56122d3-9f53-43ac-8eed-e0ae5bf436cf admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:33.112305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891ccc6-55b3-47cf-aede-b26ce96a34f6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:33.120349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f040f-7f37-44dd-8f39-7d4382360221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:33.122828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f040f-7f37-44dd-8f39-7d4382360221 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:33.122828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f040f-7f37-44dd-8f39-7d4382360221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:33.122828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f040f-7f37-44dd-8f39-7d4382360221 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:33.123783 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f040f-7f37-44dd-8f39-7d4382360221 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:33.126044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f040f-7f37-44dd-8f39-7d4382360221 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:33.183175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 920/1859] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:32 2022] POST /identity/v3/users => generated 292 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:33.187822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f040f-7f37-44dd-8f39-7d4382360221 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:33.211049 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f040f-7f37-44dd-8f39-7d4382360221 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:33.221308 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18fa18-9ef9-4e43-b301-c8d705ea4b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:33.222022 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18fa18-9ef9-4e43-b301-c8d705ea4b36 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:33.222330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18fa18-9ef9-4e43-b301-c8d705ea4b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:33.222672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18fa18-9ef9-4e43-b301-c8d705ea4b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:33.226160 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18fa18-9ef9-4e43-b301-c8d705ea4b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:33.229778 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18fa18-9ef9-4e43-b301-c8d705ea4b36 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:33.238882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72a84b-45f4-4432-82f8-50513235f732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:33.242372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72a84b-45f4-4432-82f8-50513235f732 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:33.243772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72a84b-45f4-4432-82f8-50513235f732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:33.245456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72a84b-45f4-4432-82f8-50513235f732 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:33.248179 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72a84b-45f4-4432-82f8-50513235f732 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:33.254408 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72a84b-45f4-4432-82f8-50513235f732 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:33.281225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 921/1860] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:36:33.283918 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18fa18-9ef9-4e43-b301-c8d705ea4b36 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:33.318857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18fa18-9ef9-4e43-b301-c8d705ea4b36 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:33.335446 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 940/1861] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:32 2022] POST /identity/v3/users => generated 289 bytes in 470 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:33.339616 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72a84b-45f4-4432-82f8-50513235f732 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:33.344373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad59b8-9c38-4923-b8ea-a69cfe04741f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:33.348016 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad59b8-9c38-4923-b8ea-a69cfe04741f admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:33.349374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad59b8-9c38-4923-b8ea-a69cfe04741f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:33.350358 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad59b8-9c38-4923-b8ea-a69cfe04741f admin admin] PATH_INFO: `/v3/projects/2811546d503e474bbdfe365e6d67a53c/users/266a80bbc9934ed295bd6e09c3b86e7f/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:33.371748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72a84b-45f4-4432-82f8-50513235f732 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:33.386531 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb1e67-b147-4d3b-ba6c-0a6b6db7692a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:33.389313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb1e67-b147-4d3b-ba6c-0a6b6db7692a admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:33.390772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb1e67-b147-4d3b-ba6c-0a6b6db7692a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:33.391199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb1e67-b147-4d3b-ba6c-0a6b6db7692a admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:33.394187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb1e67-b147-4d3b-ba6c-0a6b6db7692a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:33.397812 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb1e67-b147-4d3b-ba6c-0a6b6db7692a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:33.400018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad59b8-9c38-4923-b8ea-a69cfe04741f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2811546d503e474bbdfe365e6d67a53c, user_id=266a80bbc9934ed295bd6e09c3b86e7f, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:33.408904 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad59b8-9c38-4923-b8ea-a69cfe04741f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:33.425706 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 941/1862] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:33.440899 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb1e67-b147-4d3b-ba6c-0a6b6db7692a admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:33.455673 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 922/1863] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:33 2022] PUT /identity/v3/projects/2811546d503e474bbdfe365e6d67a53c/users/266a80bbc9934ed295bd6e09c3b86e7f/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:33.456832 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad59b8-9c38-4923-b8ea-a69cfe04741f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:33.471142 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb1e67-b147-4d3b-ba6c-0a6b6db7692a admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:33.615755 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65309530-b0cd-456a-afd7-da51c849546e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:33.616954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65309530-b0cd-456a-afd7-da51c849546e admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:33.621035 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65309530-b0cd-456a-afd7-da51c849546e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:33.621532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65309530-b0cd-456a-afd7-da51c849546e admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:33.624003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65309530-b0cd-456a-afd7-da51c849546e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:33.626142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad59b8-9c38-4923-b8ea-a69cfe04741f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:33.626491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65309530-b0cd-456a-afd7-da51c849546e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:33.638910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1c497-4319-436e-a1da-6e9f2b89ebff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:33.639944 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1c497-4319-436e-a1da-6e9f2b89ebff admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:33.640370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1c497-4319-436e-a1da-6e9f2b89ebff admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:33.640807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1c497-4319-436e-a1da-6e9f2b89ebff admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:33.642245 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1c497-4319-436e-a1da-6e9f2b89ebff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:33.647121 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 942/1864] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:33.652191 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1c497-4319-436e-a1da-6e9f2b89ebff admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:33.658829 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65309530-b0cd-456a-afd7-da51c849546e admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:33.683006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 923/1865] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:33.685544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1c497-4319-436e-a1da-6e9f2b89ebff admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:33.697945 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65309530-b0cd-456a-afd7-da51c849546e admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:33.703245 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:33.718983 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1c497-4319-436e-a1da-6e9f2b89ebff admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:33.719189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bb3b4-a461-404b-84e6-e4ca23826640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:33.723817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bb3b4-a461-404b-84e6-e4ca23826640 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:33.724245 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bb3b4-a461-404b-84e6-e4ca23826640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:33.725951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bb3b4-a461-404b-84e6-e4ca23826640 admin admin] PATH_INFO: `/v3/projects/0ea6f040d7b14d3bb45fdb45190692ba/users/8eadb70438a2414a8aa5197c78b3b272/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:33.731360 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f06271-b579-4874-9b18-c9803e1bd7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:33.735162 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f06271-b579-4874-9b18-c9803e1bd7a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:33.736839 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f06271-b579-4874-9b18-c9803e1bd7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:33.738849 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f06271-b579-4874-9b18-c9803e1bd7a4 admin admin] PATH_INFO: `/v3/projects/2811546d503e474bbdfe365e6d67a53c/users/266a80bbc9934ed295bd6e09c3b86e7f/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:33.743335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f06271-b579-4874-9b18-c9803e1bd7a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2811546d503e474bbdfe365e6d67a53c, user_id=266a80bbc9934ed295bd6e09c3b86e7f, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:33.750869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f06271-b579-4874-9b18-c9803e1bd7a4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:33.780211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:33.798310 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 924/1866] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:33 2022] PUT /identity/v3/projects/2811546d503e474bbdfe365e6d67a53c/users/266a80bbc9934ed295bd6e09c3b86e7f/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:33.802409 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0bb3b4-a461-404b-84e6-e4ca23826640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea6f040d7b14d3bb45fdb45190692ba, user_id=8eadb70438a2414a8aa5197c78b3b272, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:33.809077 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0bb3b4-a461-404b-84e6-e4ca23826640 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:33.844086 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 943/1867] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:33 2022] PUT /identity/v3/projects/0ea6f040d7b14d3bb45fdb45190692ba/users/8eadb70438a2414a8aa5197c78b3b272/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:33.857124 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc803575-cec0-49bf-8fab-c3b9707e5352 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:33.857477 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc803575-cec0-49bf-8fab-c3b9707e5352 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:33.858136 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc803575-cec0-49bf-8fab-c3b9707e5352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:33.906709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bc803575-cec0-49bf-8fab-c3b9707e5352 None None] MFA Rules not processed for user `8eadb70438a2414a8aa5197c78b3b272`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:33.908815 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bc803575-cec0-49bf-8fab-c3b9707e5352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:34.060169 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 925/1868] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:34.072759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bb3b4-a461-404b-84e6-e4ca23826640 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:34.074840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-9b0bb3b4-a461-404b-84e6-e4ca23826640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:34.256098 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bb3b4-a461-404b-84e6-e4ca23826640 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:34.274317 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95203b21-e8eb-4946-ab30-8fc5b400d995 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36de21e0b3de48dabe0f27a9db8a37aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:34.275335 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95203b21-e8eb-4946-ab30-8fc5b400d995 service glance] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:34.275688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95203b21-e8eb-4946-ab30-8fc5b400d995 service glance] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:34.276133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95203b21-e8eb-4946-ab30-8fc5b400d995 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:34.323057 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95203b21-e8eb-4946-ab30-8fc5b400d995 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:34.324540 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95203b21-e8eb-4946-ab30-8fc5b400d995 service glance] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:34.353975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343916d-eafe-4b41-b725-c25979191f31 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:34.354665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343916d-eafe-4b41-b725-c25979191f31 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:34.354665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343916d-eafe-4b41-b725-c25979191f31 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:34.356066 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 926/1869] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 22:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:36:34.369359 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 944/1870] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:34.427365 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0de2a-2551-404e-8922-350b432e89f2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:34.427691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0de2a-2551-404e-8922-350b432e89f2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:34.427874 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0de2a-2551-404e-8922-350b432e89f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:34.506012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3de0de2a-2551-404e-8922-350b432e89f2 None None] MFA Rules not processed for user `101864afd59543e699189917391e6ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:34.511970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3de0de2a-2551-404e-8922-350b432e89f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:34.719161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 927/1871] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 22:36:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:34.941255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95203b21-e8eb-4946-ab30-8fc5b400d995 service glance] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:34.943522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-95203b21-e8eb-4946-ab30-8fc5b400d995 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:35.007751 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95203b21-e8eb-4946-ab30-8fc5b400d995 service glance] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:35.061701 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628dee1f-18a0-4737-973a-200304fb864d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101864afd59543e699189917391e6ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:35.062621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628dee1f-18a0-4737-973a-200304fb864d service placement] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:35.069051 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628dee1f-18a0-4737-973a-200304fb864d service placement] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:35.069617 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628dee1f-18a0-4737-973a-200304fb864d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:35.099735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628dee1f-18a0-4737-973a-200304fb864d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:35.101387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628dee1f-18a0-4737-973a-200304fb864d service placement] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:35.121257 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 945/1872] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:35.610479 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0de2a-2551-404e-8922-350b432e89f2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:35.612105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3de0de2a-2551-404e-8922-350b432e89f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:35.719129 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0de2a-2551-404e-8922-350b432e89f2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:35.724310 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04ce11-35a3-4ade-8ebd-fd600acd6ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:35.725451 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04ce11-35a3-4ade-8ebd-fd600acd6ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:35.725909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04ce11-35a3-4ade-8ebd-fd600acd6ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:35.726438 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04ce11-35a3-4ade-8ebd-fd600acd6ee6 admin admin] PATH_INFO: `/v3/users/21fe10e2f309460d8cb01f9cd3fbf450` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:35.727964 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04ce11-35a3-4ade-8ebd-fd600acd6ee6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21fe10e2f309460d8cb01f9cd3fbf450)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:35.730769 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04ce11-35a3-4ade-8ebd-fd600acd6ee6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:35.755178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:35.817660 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5a04ce11-35a3-4ade-8ebd-fd600acd6ee6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21fe10e2f309460d8cb01f9cd3fbf450'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:35.831343 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5a04ce11-35a3-4ade-8ebd-fd600acd6ee6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21fe10e2f309460d8cb01f9cd3fbf450'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:35.844343 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5a04ce11-35a3-4ade-8ebd-fd600acd6ee6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21fe10e2f309460d8cb01f9cd3fbf450'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:35.845975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5a04ce11-35a3-4ade-8ebd-fd600acd6ee6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21fe10e2f309460d8cb01f9cd3fbf450'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:35.848032 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-5a04ce11-35a3-4ade-8ebd-fd600acd6ee6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21fe10e2f309460d8cb01f9cd3fbf450'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:35.881647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 928/1873] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:36:35 2022] DELETE /identity/v3/users/21fe10e2f309460d8cb01f9cd3fbf450 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:35.886988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b0f02-c4a3-42e1-87da-ded68db16e72 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:35.887543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b0f02-c4a3-42e1-87da-ded68db16e72 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:35.891499 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b0f02-c4a3-42e1-87da-ded68db16e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:35.913701 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-9f4b0f02-c4a3-42e1-87da-ded68db16e72 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:35.915612 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-9f4b0f02-c4a3-42e1-87da-ded68db16e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:36.025544 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 946/1874] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:36.063807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04ce11-35a3-4ade-8ebd-fd600acd6ee6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:36.066047 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5a04ce11-35a3-4ade-8ebd-fd600acd6ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:36.210684 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04ce11-35a3-4ade-8ebd-fd600acd6ee6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:36.223754 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc56b0-5884-418b-b078-be0bcbb16160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:36.230224 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc56b0-5884-418b-b078-be0bcbb16160 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:36.230989 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc56b0-5884-418b-b078-be0bcbb16160 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:36.233095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc56b0-5884-418b-b078-be0bcbb16160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:36.268367 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc56b0-5884-418b-b078-be0bcbb16160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:36.269934 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc56b0-5884-418b-b078-be0bcbb16160 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:36.291806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 929/1875] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:36.814010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4835da2-dbd5-44a6-9a43-bd71f7afb3bf None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:36.815572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4835da2-dbd5-44a6-9a43-bd71f7afb3bf None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:36.816293 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4835da2-dbd5-44a6-9a43-bd71f7afb3bf None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:36.818093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 947/1876] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 22:36:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:36:36.842284 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f0e44-b0ab-4919-9274-f9ffb6f50985 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:36.842884 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f0e44-b0ab-4919-9274-f9ffb6f50985 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:36.843138 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f0e44-b0ab-4919-9274-f9ffb6f50985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:36.868207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-180f0e44-b0ab-4919-9274-f9ffb6f50985 None None] MFA Rules not processed for user `101864afd59543e699189917391e6ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:36.869917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-180f0e44-b0ab-4919-9274-f9ffb6f50985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:36.969264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 930/1877] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 22:36:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:36.986983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4835da2-dbd5-44a6-9a43-bd71f7afb3bf None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:36.988945 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d4835da2-dbd5-44a6-9a43-bd71f7afb3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:36.996748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f0e44-b0ab-4919-9274-f9ffb6f50985 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:37.011866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-180f0e44-b0ab-4919-9274-f9ffb6f50985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:37.041396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4835da2-dbd5-44a6-9a43-bd71f7afb3bf None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:37.053922 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f0e44-b0ab-4919-9274-f9ffb6f50985 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:37.058542 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a193b1-3d69-4dba-8ace-cfe417594cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:37.061168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a193b1-3d69-4dba-8ace-cfe417594cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:37.061709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a193b1-3d69-4dba-8ace-cfe417594cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:37.062213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a193b1-3d69-4dba-8ace-cfe417594cf8 admin admin] PATH_INFO: `/v3/users/13f6decd0e80403e95cc609900b7396b` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:37.063749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a193b1-3d69-4dba-8ace-cfe417594cf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13f6decd0e80403e95cc609900b7396b)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:37.066062 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a193b1-3d69-4dba-8ace-cfe417594cf8 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:37.070412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec254e1e-db62-4781-9271-281641596988 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101864afd59543e699189917391e6ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:37.071312 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec254e1e-db62-4781-9271-281641596988 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:37.071612 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec254e1e-db62-4781-9271-281641596988 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:37.071906 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec254e1e-db62-4781-9271-281641596988 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:37.082040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec254e1e-db62-4781-9271-281641596988 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:37.083879 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec254e1e-db62-4781-9271-281641596988 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:37.114296 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 931/1878] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:37.116157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88735544-f2b8-42c2-a147-cc792f1f12d0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:37.116523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88735544-f2b8-42c2-a147-cc792f1f12d0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:37.116838 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88735544-f2b8-42c2-a147-cc792f1f12d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:37.178753 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-88735544-f2b8-42c2-a147-cc792f1f12d0 None None] MFA Rules not processed for user `266a80bbc9934ed295bd6e09c3b86e7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:37.180238 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-88735544-f2b8-42c2-a147-cc792f1f12d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:37.183719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e8a193b1-3d69-4dba-8ace-cfe417594cf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13f6decd0e80403e95cc609900b7396b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:37.192360 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:37.210872 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e8a193b1-3d69-4dba-8ace-cfe417594cf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13f6decd0e80403e95cc609900b7396b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:37.223844 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e8a193b1-3d69-4dba-8ace-cfe417594cf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13f6decd0e80403e95cc609900b7396b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:37.248246 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:37.257903 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e8a193b1-3d69-4dba-8ace-cfe417594cf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13f6decd0e80403e95cc609900b7396b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:37.260718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e8a193b1-3d69-4dba-8ace-cfe417594cf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13f6decd0e80403e95cc609900b7396b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:37.284931 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 948/1879] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:36:36 2022] DELETE /identity/v3/users/13f6decd0e80403e95cc609900b7396b => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:37.373326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 932/1880] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:37 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:37.389385 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a193b1-3d69-4dba-8ace-cfe417594cf8 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:37.391294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e8a193b1-3d69-4dba-8ace-cfe417594cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:37.442474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88735544-f2b8-42c2-a147-cc792f1f12d0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:37.444166 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-88735544-f2b8-42c2-a147-cc792f1f12d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:37.468764 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88735544-f2b8-42c2-a147-cc792f1f12d0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:37.518009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a193b1-3d69-4dba-8ace-cfe417594cf8 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:37.532402 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592208d0-e5a8-4bd3-a90d-d80cfe424dca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:37.534193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592208d0-e5a8-4bd3-a90d-d80cfe424dca service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:37.536508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592208d0-e5a8-4bd3-a90d-d80cfe424dca service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:37.543790 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592208d0-e5a8-4bd3-a90d-d80cfe424dca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:37.552977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-592208d0-e5a8-4bd3-a90d-d80cfe424dca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:37.614714 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7f597-9885-4abe-ab7b-934ca349b342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:37.615656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7f597-9885-4abe-ab7b-934ca349b342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:37.616030 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7f597-9885-4abe-ab7b-934ca349b342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:37.616258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7f597-9885-4abe-ab7b-934ca349b342 admin admin] PATH_INFO: `/v3/projects/3d69a6f283a4406cb4a522a44849363f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:37.617779 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7f597-9885-4abe-ab7b-934ca349b342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d69a6f283a4406cb4a522a44849363f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:37.620340 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7f597-9885-4abe-ab7b-934ca349b342 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:37.696980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592208d0-e5a8-4bd3-a90d-d80cfe424dca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:37.699768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592208d0-e5a8-4bd3-a90d-d80cfe424dca service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:37.737066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 949/1881] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:37.739389 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592208d0-e5a8-4bd3-a90d-d80cfe424dca service cinder] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:37.742816 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-592208d0-e5a8-4bd3-a90d-d80cfe424dca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:37.754140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d7a7f597-9885-4abe-ab7b-934ca349b342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d69a6f283a4406cb4a522a44849363f', 'request_id': 'req-d7a7f597-9885-4abe-ab7b-934ca349b342'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:37.770590 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d7a7f597-9885-4abe-ab7b-934ca349b342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d69a6f283a4406cb4a522a44849363f', 'request_id': 'req-d7a7f597-9885-4abe-ab7b-934ca349b342'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:37.782219 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592208d0-e5a8-4bd3-a90d-d80cfe424dca service cinder] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:37.786792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d7a7f597-9885-4abe-ab7b-934ca349b342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d69a6f283a4406cb4a522a44849363f', 'request_id': 'req-d7a7f597-9885-4abe-ab7b-934ca349b342'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:37.804108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d7a7f597-9885-4abe-ab7b-934ca349b342 admin admin] The token cache is being invalidate because project 3d69a6f283a4406cb4a522a44849363f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:36:37.804841 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d7a7f597-9885-4abe-ab7b-934ca349b342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:37.805084 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d7a7f597-9885-4abe-ab7b-934ca349b342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:37.807065 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 933/1882] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:36:37 2022] DELETE /identity/v3/projects/3d69a6f283a4406cb4a522a44849363f => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:36:37.819341 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e072f4-b661-43ee-9318-f62988a105bf None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:37.820117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e072f4-b661-43ee-9318-f62988a105bf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:37.820510 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e072f4-b661-43ee-9318-f62988a105bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:37.860170 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-03e072f4-b661-43ee-9318-f62988a105bf None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:37.861572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-03e072f4-b661-43ee-9318-f62988a105bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:37.925376 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:37.932878 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a2cee-efd5-4f13-b48a-991da92f2edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:37.937282 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a2cee-efd5-4f13-b48a-991da92f2edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:37.942519 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a2cee-efd5-4f13-b48a-991da92f2edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:37.943238 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a2cee-efd5-4f13-b48a-991da92f2edb admin admin] PATH_INFO: `/v3/projects/1f2a574c6e764b53bf8144badf2ab027` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:37.947976 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a2cee-efd5-4f13-b48a-991da92f2edb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f2a574c6e764b53bf8144badf2ab027)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:37.951610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a2cee-efd5-4f13-b48a-991da92f2edb admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:38.018796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 934/1883] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:38.020934 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ceb62-4b12-4969-a907-7bd708b67c56 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:38.021536 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ceb62-4b12-4969-a907-7bd708b67c56 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:38.022162 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ceb62-4b12-4969-a907-7bd708b67c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:38.053432 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1a3ceb62-4b12-4969-a907-7bd708b67c56 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:38.055058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1a3ceb62-4b12-4969-a907-7bd708b67c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:38.066227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 935/1884] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:38.071427 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ceb62-4b12-4969-a907-7bd708b67c56 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:38.085348 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ceb62-4b12-4969-a907-7bd708b67c56 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:38.101567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-be2a2cee-efd5-4f13-b48a-991da92f2edb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f2a574c6e764b53bf8144badf2ab027', 'request_id': 'req-be2a2cee-efd5-4f13-b48a-991da92f2edb'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:38.102767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:38.117604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-be2a2cee-efd5-4f13-b48a-991da92f2edb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f2a574c6e764b53bf8144badf2ab027', 'request_id': 'req-be2a2cee-efd5-4f13-b48a-991da92f2edb'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:38.140087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-be2a2cee-efd5-4f13-b48a-991da92f2edb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f2a574c6e764b53bf8144badf2ab027', 'request_id': 'req-be2a2cee-efd5-4f13-b48a-991da92f2edb'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:38.161076 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-be2a2cee-efd5-4f13-b48a-991da92f2edb admin admin] The token cache is being invalidate because project 1f2a574c6e764b53bf8144badf2ab027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:36:38.162040 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-be2a2cee-efd5-4f13-b48a-991da92f2edb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:38.164015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-be2a2cee-efd5-4f13-b48a-991da92f2edb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:38.167797 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 950/1885] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:36:37 2022] DELETE /identity/v3/projects/1f2a574c6e764b53bf8144badf2ab027 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:38.172824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a2cee-efd5-4f13-b48a-991da92f2edb admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:38.174368 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-be2a2cee-efd5-4f13-b48a-991da92f2edb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:38.181822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b446472-1312-4819-ad1f-0ad1e4a8a9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:38.182852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b446472-1312-4819-ad1f-0ad1e4a8a9ab admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:38.182852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b446472-1312-4819-ad1f-0ad1e4a8a9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:38.183154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b446472-1312-4819-ad1f-0ad1e4a8a9ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:38.184008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b446472-1312-4819-ad1f-0ad1e4a8a9ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:36:38.184322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b446472-1312-4819-ad1f-0ad1e4a8a9ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:38.192328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b446472-1312-4819-ad1f-0ad1e4a8a9ab admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:38.222746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 936/1886] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:36:38.228706 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b446472-1312-4819-ad1f-0ad1e4a8a9ab admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:38.236238 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2b446472-1312-4819-ad1f-0ad1e4a8a9ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:38.257207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b446472-1312-4819-ad1f-0ad1e4a8a9ab admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:38.266973 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278c21b-63e7-4801-b2dd-df1954e45fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:38.266973 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278c21b-63e7-4801-b2dd-df1954e45fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:38.266973 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278c21b-63e7-4801-b2dd-df1954e45fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:38.266973 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278c21b-63e7-4801-b2dd-df1954e45fd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:38.267621 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278c21b-63e7-4801-b2dd-df1954e45fd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:38.267902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278c21b-63e7-4801-b2dd-df1954e45fd9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:38.305250 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 937/1887] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:38 2022] POST /identity/v3/projects => generated 361 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:38.310473 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278c21b-63e7-4801-b2dd-df1954e45fd9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:38.320689 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a2cee-efd5-4f13-b48a-991da92f2edb admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:38.383637 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278c21b-63e7-4801-b2dd-df1954e45fd9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:38.389363 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ab387-58e4-4757-87e4-53dab1e77879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:38.390152 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ab387-58e4-4757-87e4-53dab1e77879 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:38.390442 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ab387-58e4-4757-87e4-53dab1e77879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:38.390714 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ab387-58e4-4757-87e4-53dab1e77879 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:38.391518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ab387-58e4-4757-87e4-53dab1e77879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:38.393453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ab387-58e4-4757-87e4-53dab1e77879 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:38.396353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dcc452-e70e-4f35-b73c-327b29260e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:38.397266 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcc452-e70e-4f35-b73c-327b29260e14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:38.397555 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcc452-e70e-4f35-b73c-327b29260e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:38.397987 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcc452-e70e-4f35-b73c-327b29260e14 admin admin] PATH_INFO: `/v3/users/23c20a474a1341acb8a1b2c71a93addf` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:38.399303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dcc452-e70e-4f35-b73c-327b29260e14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c20a474a1341acb8a1b2c71a93addf)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:38.401651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dcc452-e70e-4f35-b73c-327b29260e14 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:38.415336 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 938/1888] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:38 2022] POST /identity/v3/users => generated 281 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:36:38.420546 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ab387-58e4-4757-87e4-53dab1e77879 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:38.430607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ab387-58e4-4757-87e4-53dab1e77879 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:38.437241 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c352803-54fd-49cc-aafb-f0c4bd1ec8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:38.438000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c352803-54fd-49cc-aafb-f0c4bd1ec8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:38.438276 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c352803-54fd-49cc-aafb-f0c4bd1ec8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:38.438553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c352803-54fd-49cc-aafb-f0c4bd1ec8e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:38.439390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c352803-54fd-49cc-aafb-f0c4bd1ec8e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:38.441100 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c352803-54fd-49cc-aafb-f0c4bd1ec8e1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:38.456542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 939/1889] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:36:38.458623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c352803-54fd-49cc-aafb-f0c4bd1ec8e1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:38.473875 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e5dcc452-e70e-4f35-b73c-327b29260e14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c20a474a1341acb8a1b2c71a93addf'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:38.474199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c352803-54fd-49cc-aafb-f0c4bd1ec8e1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:38.479302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5c6df-ec6e-4ce2-9455-edcbf94f4006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:38.480123 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5c6df-ec6e-4ce2-9455-edcbf94f4006 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:38.480422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5c6df-ec6e-4ce2-9455-edcbf94f4006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:38.480712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5c6df-ec6e-4ce2-9455-edcbf94f4006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:38.482061 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5c6df-ec6e-4ce2-9455-edcbf94f4006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:38.483982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5c6df-ec6e-4ce2-9455-edcbf94f4006 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:38.489632 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e5dcc452-e70e-4f35-b73c-327b29260e14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c20a474a1341acb8a1b2c71a93addf'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:38.495270 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 940/1890] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:38.500107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5c6df-ec6e-4ce2-9455-edcbf94f4006 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:38.501579 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e5dcc452-e70e-4f35-b73c-327b29260e14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c20a474a1341acb8a1b2c71a93addf'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:38.511035 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e5dcc452-e70e-4f35-b73c-327b29260e14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c20a474a1341acb8a1b2c71a93addf'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:38.511822 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e5dcc452-e70e-4f35-b73c-327b29260e14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c20a474a1341acb8a1b2c71a93addf'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:38.513620 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5c6df-ec6e-4ce2-9455-edcbf94f4006 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:38.519678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 951/1891] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:36:38 2022] DELETE /identity/v3/users/23c20a474a1341acb8a1b2c71a93addf => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:38.522616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db7bf0-10cc-4992-82a4-7235c73541f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:38.523646 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db7bf0-10cc-4992-82a4-7235c73541f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:38.523940 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db7bf0-10cc-4992-82a4-7235c73541f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:38.524228 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db7bf0-10cc-4992-82a4-7235c73541f2 admin admin] PATH_INFO: `/v3/projects/64cad109fe5449aea36e5222ab51042c/users/ea953d70389f4a8d8df29069fafb847a/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:38.528887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:38.546049 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db7bf0-10cc-4992-82a4-7235c73541f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64cad109fe5449aea36e5222ab51042c, user_id=ea953d70389f4a8d8df29069fafb847a, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:38.549194 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db7bf0-10cc-4992-82a4-7235c73541f2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:38.571013 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 941/1892] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:38 2022] PUT /identity/v3/projects/64cad109fe5449aea36e5222ab51042c/users/ea953d70389f4a8d8df29069fafb847a/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:38.579997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3e91a-48ba-4b39-bb7f-bc65064a1400 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:38.580727 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3e91a-48ba-4b39-bb7f-bc65064a1400 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:38.581145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3e91a-48ba-4b39-bb7f-bc65064a1400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:38.595437 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:38.631284 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-8fb3e91a-48ba-4b39-bb7f-bc65064a1400 None None] MFA Rules not processed for user `ea953d70389f4a8d8df29069fafb847a`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:38.632663 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8fb3e91a-48ba-4b39-bb7f-bc65064a1400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:38.743105 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 952/1893] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:38.791532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db7bf0-10cc-4992-82a4-7235c73541f2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:38.793389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-48db7bf0-10cc-4992-82a4-7235c73541f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:38.894447 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db7bf0-10cc-4992-82a4-7235c73541f2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:38.898097 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3c580-cdaa-4636-9396-d8fd409e886b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:38.898756 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3c580-cdaa-4636-9396-d8fd409e886b service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:38.899090 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3c580-cdaa-4636-9396-d8fd409e886b service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:38.899371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3c580-cdaa-4636-9396-d8fd409e886b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:38.911211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3c580-cdaa-4636-9396-d8fd409e886b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:38.912643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3c580-cdaa-4636-9396-d8fd409e886b service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:38.928081 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 942/1894] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:38.945414 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3e91a-48ba-4b39-bb7f-bc65064a1400 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:38.947087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8fb3e91a-48ba-4b39-bb7f-bc65064a1400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:39.062882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3e91a-48ba-4b39-bb7f-bc65064a1400 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:39.069452 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46309e-e8f6-4370-9657-dd7d945269d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:39.069881 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46309e-e8f6-4370-9657-dd7d945269d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:39.070181 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46309e-e8f6-4370-9657-dd7d945269d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:39.070472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46309e-e8f6-4370-9657-dd7d945269d3 admin admin] PATH_INFO: `/v3/users/ea953d70389f4a8d8df29069fafb847a` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:39.071733 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46309e-e8f6-4370-9657-dd7d945269d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea953d70389f4a8d8df29069fafb847a)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:39.073707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46309e-e8f6-4370-9657-dd7d945269d3 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:39.155059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-bc46309e-e8f6-4370-9657-dd7d945269d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea953d70389f4a8d8df29069fafb847a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:39.183201 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-bc46309e-e8f6-4370-9657-dd7d945269d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea953d70389f4a8d8df29069fafb847a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:39.193988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-bc46309e-e8f6-4370-9657-dd7d945269d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea953d70389f4a8d8df29069fafb847a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:39.206653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-bc46309e-e8f6-4370-9657-dd7d945269d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea953d70389f4a8d8df29069fafb847a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:39.207457 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-bc46309e-e8f6-4370-9657-dd7d945269d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea953d70389f4a8d8df29069fafb847a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:39.215937 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 953/1895] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:36:38 2022] DELETE /identity/v3/users/ea953d70389f4a8d8df29069fafb847a => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:39.221246 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb31096-b9a2-40c7-a86e-a89807b0de2e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:39.221546 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb31096-b9a2-40c7-a86e-a89807b0de2e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:39.221839 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb31096-b9a2-40c7-a86e-a89807b0de2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:39.278130 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ecb31096-b9a2-40c7-a86e-a89807b0de2e None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:39.281222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46309e-e8f6-4370-9657-dd7d945269d3 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:39.282704 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-bc46309e-e8f6-4370-9657-dd7d945269d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:39.288579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ecb31096-b9a2-40c7-a86e-a89807b0de2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:39.333645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:39.441136 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46309e-e8f6-4370-9657-dd7d945269d3 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:39.448036 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f59456-eb2c-47b0-822b-d9b1d708d661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:39.449301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f59456-eb2c-47b0-822b-d9b1d708d661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:39.449660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f59456-eb2c-47b0-822b-d9b1d708d661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:39.451009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f59456-eb2c-47b0-822b-d9b1d708d661 admin admin] PATH_INFO: `/v3/projects/61456fd5f2cc44c8aa7ba760ae870ce9` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:39.457344 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f59456-eb2c-47b0-822b-d9b1d708d661 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61456fd5f2cc44c8aa7ba760ae870ce9)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:39.457344 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f59456-eb2c-47b0-822b-d9b1d708d661 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:39.463029 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 943/1896] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:39.467762 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb31096-b9a2-40c7-a86e-a89807b0de2e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:39.471772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ecb31096-b9a2-40c7-a86e-a89807b0de2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:39.548472 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb31096-b9a2-40c7-a86e-a89807b0de2e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:39.562070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-40f59456-eb2c-47b0-822b-d9b1d708d661 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61456fd5f2cc44c8aa7ba760ae870ce9', 'request_id': 'req-40f59456-eb2c-47b0-822b-d9b1d708d661'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:39.581667 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-40f59456-eb2c-47b0-822b-d9b1d708d661 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61456fd5f2cc44c8aa7ba760ae870ce9', 'request_id': 'req-40f59456-eb2c-47b0-822b-d9b1d708d661'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:39.599737 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-40f59456-eb2c-47b0-822b-d9b1d708d661 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61456fd5f2cc44c8aa7ba760ae870ce9', 'request_id': 'req-40f59456-eb2c-47b0-822b-d9b1d708d661'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:39.632411 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-40f59456-eb2c-47b0-822b-d9b1d708d661 admin admin] The token cache is being invalidate because project 61456fd5f2cc44c8aa7ba760ae870ce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:36:39.632832 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-40f59456-eb2c-47b0-822b-d9b1d708d661 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:39.635324 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-40f59456-eb2c-47b0-822b-d9b1d708d661 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:39.641126 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 954/1897] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:36:39 2022] DELETE /identity/v3/projects/61456fd5f2cc44c8aa7ba760ae870ce9 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:36:39.641126 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f59456-eb2c-47b0-822b-d9b1d708d661 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:39.641126 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-40f59456-eb2c-47b0-822b-d9b1d708d661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:39.674965 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:39.703370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f739df-eb6d-428a-a6fc-9563f63c85a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:39.704053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f739df-eb6d-428a-a6fc-9563f63c85a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:39.704345 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f739df-eb6d-428a-a6fc-9563f63c85a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:39.704634 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f739df-eb6d-428a-a6fc-9563f63c85a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:39.706519 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-44f739df-eb6d-428a-a6fc-9563f63c85a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:39.767035 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f59456-eb2c-47b0-822b-d9b1d708d661 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:39.783972 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76640b-479b-4397-9b1e-f8579267b4c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:39.784950 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76640b-479b-4397-9b1e-f8579267b4c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:39.785419 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76640b-479b-4397-9b1e-f8579267b4c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:39.785707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76640b-479b-4397-9b1e-f8579267b4c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:39.787639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-cb76640b-479b-4397-9b1e-f8579267b4c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:39.820364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f739df-eb6d-428a-a6fc-9563f63c85a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:39.824660 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f739df-eb6d-428a-a6fc-9563f63c85a7 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:39.864984 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 944/1898] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:39 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:39.868253 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d669f11-672d-4b2f-9452-42ee179f49d9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:39.868549 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d669f11-672d-4b2f-9452-42ee179f49d9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:39.868834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d669f11-672d-4b2f-9452-42ee179f49d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:39.905664 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:39.916418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8d669f11-672d-4b2f-9452-42ee179f49d9 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:39.917936 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8d669f11-672d-4b2f-9452-42ee179f49d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:39.923099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76640b-479b-4397-9b1e-f8579267b4c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:39.927134 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76640b-479b-4397-9b1e-f8579267b4c2 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:39.944962 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 945/1899] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:39.944962 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f2e70-de46-45ef-82fb-35f35cd73ae2 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:39.944962 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f2e70-de46-45ef-82fb-35f35cd73ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:39.944962 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f2e70-de46-45ef-82fb-35f35cd73ae2 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:39.944962 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 946/1900] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 22:36:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:36:39.947462 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7b4c0-c7ae-4412-83b9-24a1e2668b9e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:39.947758 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7b4c0-c7ae-4412-83b9-24a1e2668b9e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:39.948044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7b4c0-c7ae-4412-83b9-24a1e2668b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:39.952818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 955/1901] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:39.980478 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-eeb7b4c0-c7ae-4412-83b9-24a1e2668b9e None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:39.982938 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-eeb7b4c0-c7ae-4412-83b9-24a1e2668b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:39.995005 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 947/1902] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:39.999676 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76640b-479b-4397-9b1e-f8579267b4c2 service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:40.000809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6a8fa-6004-4ad1-9755-92759cdf2d50 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:40.003168 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6a8fa-6004-4ad1-9755-92759cdf2d50 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:40.003860 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6a8fa-6004-4ad1-9755-92759cdf2d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:40.019142 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76640b-479b-4397-9b1e-f8579267b4c2 service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:40.025361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66e7f4-2b92-4a4b-934c-6597915c6fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:40.026249 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66e7f4-2b92-4a4b-934c-6597915c6fde admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:40.026687 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66e7f4-2b92-4a4b-934c-6597915c6fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:40.027170 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66e7f4-2b92-4a4b-934c-6597915c6fde admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:40.028266 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66e7f4-2b92-4a4b-934c-6597915c6fde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:36:40.030608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66e7f4-2b92-4a4b-934c-6597915c6fde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:40.032788 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66e7f4-2b92-4a4b-934c-6597915c6fde admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:40.049338 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 956/1903] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:36:40.056131 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66e7f4-2b92-4a4b-934c-6597915c6fde admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:40.065266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bcb6a8fa-6004-4ad1-9755-92759cdf2d50 None None] MFA Rules not processed for user `35d686923ef541819de3c77847d38511`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:40.070454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bcb6a8fa-6004-4ad1-9755-92759cdf2d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:40.074450 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66e7f4-2b92-4a4b-934c-6597915c6fde admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:40.080743 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f719009-ee3f-4470-908a-5233615c7fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:40.082666 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f719009-ee3f-4470-908a-5233615c7fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:40.083117 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f719009-ee3f-4470-908a-5233615c7fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:40.083540 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f719009-ee3f-4470-908a-5233615c7fe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:40.084403 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f719009-ee3f-4470-908a-5233615c7fe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:40.086785 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f719009-ee3f-4470-908a-5233615c7fe5 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:40.163640 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 957/1904] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:40 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:40.164939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f719009-ee3f-4470-908a-5233615c7fe5 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:40.236578 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 948/1905] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 22:36:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:40.325520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f719009-ee3f-4470-908a-5233615c7fe5 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:40.331236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357609a6-54a7-4eae-9799-48a0152d157f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:40.332250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357609a6-54a7-4eae-9799-48a0152d157f admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:40.332730 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357609a6-54a7-4eae-9799-48a0152d157f admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:40.333076 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357609a6-54a7-4eae-9799-48a0152d157f admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:40.334074 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357609a6-54a7-4eae-9799-48a0152d157f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:40.336790 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357609a6-54a7-4eae-9799-48a0152d157f admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:40.386387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6a8fa-6004-4ad1-9755-92759cdf2d50 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:40.386643 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 958/1906] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:40 2022] POST /identity/v3/users => generated 286 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:40.405553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6a8fa-6004-4ad1-9755-92759cdf2d50 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:40.417175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce23811-8009-45fd-a7b0-ec0c8782a125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:40.420974 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce23811-8009-45fd-a7b0-ec0c8782a125 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:40.422105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce23811-8009-45fd-a7b0-ec0c8782a125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:40.423907 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce23811-8009-45fd-a7b0-ec0c8782a125 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:40.426710 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce23811-8009-45fd-a7b0-ec0c8782a125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:40.436390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce23811-8009-45fd-a7b0-ec0c8782a125 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:40.463029 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 949/1907] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:36:40.468976 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357609a6-54a7-4eae-9799-48a0152d157f admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:40.490347 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357609a6-54a7-4eae-9799-48a0152d157f admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:40.499998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273c015-8cec-4c52-a37d-6002a4b8e6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:40.501314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273c015-8cec-4c52-a37d-6002a4b8e6db admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:40.501867 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273c015-8cec-4c52-a37d-6002a4b8e6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:40.502339 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273c015-8cec-4c52-a37d-6002a4b8e6db admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:40.503724 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273c015-8cec-4c52-a37d-6002a4b8e6db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:40.508294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273c015-8cec-4c52-a37d-6002a4b8e6db admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:40.527664 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce23811-8009-45fd-a7b0-ec0c8782a125 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:40.532137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2ce23811-8009-45fd-a7b0-ec0c8782a125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:40.548074 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 959/1908] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:40.557807 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273c015-8cec-4c52-a37d-6002a4b8e6db admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:40.571312 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273c015-8cec-4c52-a37d-6002a4b8e6db admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:40.579935 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dab9cb-51f3-4841-b913-d02f61a317ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:40.580975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dab9cb-51f3-4841-b913-d02f61a317ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:40.589111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dab9cb-51f3-4841-b913-d02f61a317ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:40.589821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dab9cb-51f3-4841-b913-d02f61a317ee admin admin] PATH_INFO: `/v3/projects/c963ec19d64b44758a96a22c7c8c3450/users/43596e8c32d041a0926976c255867a05/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:40.612553 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dab9cb-51f3-4841-b913-d02f61a317ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=c963ec19d64b44758a96a22c7c8c3450, user_id=43596e8c32d041a0926976c255867a05, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:40.616028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dab9cb-51f3-4841-b913-d02f61a317ee admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:40.633334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:40.636926 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 960/1909] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:40 2022] PUT /identity/v3/projects/c963ec19d64b44758a96a22c7c8c3450/users/43596e8c32d041a0926976c255867a05/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:40.649720 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f574b4-ddb2-4acd-ae95-718113b597da None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:40.650391 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f574b4-ddb2-4acd-ae95-718113b597da None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:40.651281 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f574b4-ddb2-4acd-ae95-718113b597da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:40.658907 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce23811-8009-45fd-a7b0-ec0c8782a125 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:40.691453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-e3f574b4-ddb2-4acd-ae95-718113b597da None None] MFA Rules not processed for user `43596e8c32d041a0926976c255867a05`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:40.692975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e3f574b4-ddb2-4acd-ae95-718113b597da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:40.769101 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12431b-9a35-4710-8f69-7272b79525bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:40.770758 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12431b-9a35-4710-8f69-7272b79525bd service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:40.771224 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12431b-9a35-4710-8f69-7272b79525bd service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:40.771970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12431b-9a35-4710-8f69-7272b79525bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:40.785318 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12431b-9a35-4710-8f69-7272b79525bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:40.787775 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12431b-9a35-4710-8f69-7272b79525bd service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:40.826088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 950/1910] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:36:40.827333 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12431b-9a35-4710-8f69-7272b79525bd service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:40.833069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8e12431b-9a35-4710-8f69-7272b79525bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:40.913010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 961/1911] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:40.917587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f574b4-ddb2-4acd-ae95-718113b597da None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:40.927026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:41.075175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12431b-9a35-4710-8f69-7272b79525bd service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:41.088715 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51569174-3f1c-4947-b585-fd6031dd6fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:41.089815 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51569174-3f1c-4947-b585-fd6031dd6fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:41.093557 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51569174-3f1c-4947-b585-fd6031dd6fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:41.094438 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51569174-3f1c-4947-b585-fd6031dd6fb2 admin admin] PATH_INFO: `/v3/projects/64cad109fe5449aea36e5222ab51042c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:41.096941 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51569174-3f1c-4947-b585-fd6031dd6fb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64cad109fe5449aea36e5222ab51042c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:41.107314 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51569174-3f1c-4947-b585-fd6031dd6fb2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:41.216567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f574b4-ddb2-4acd-ae95-718113b597da None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:41.227702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-51569174-3f1c-4947-b585-fd6031dd6fb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64cad109fe5449aea36e5222ab51042c', 'request_id': 'req-51569174-3f1c-4947-b585-fd6031dd6fb2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:41.240276 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-51569174-3f1c-4947-b585-fd6031dd6fb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64cad109fe5449aea36e5222ab51042c', 'request_id': 'req-51569174-3f1c-4947-b585-fd6031dd6fb2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:41.262607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-51569174-3f1c-4947-b585-fd6031dd6fb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64cad109fe5449aea36e5222ab51042c', 'request_id': 'req-51569174-3f1c-4947-b585-fd6031dd6fb2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:41.296562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-51569174-3f1c-4947-b585-fd6031dd6fb2 admin admin] The token cache is being invalidate because project 64cad109fe5449aea36e5222ab51042c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:36:41.303861 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-51569174-3f1c-4947-b585-fd6031dd6fb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:41.307123 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-51569174-3f1c-4947-b585-fd6031dd6fb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:36:41.309237 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 951/1912] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:36:40 2022] DELETE /identity/v3/projects/64cad109fe5449aea36e5222ab51042c => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:41.314525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80512b26-efcd-4543-a068-33975ffc10e4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:41.314824 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80512b26-efcd-4543-a068-33975ffc10e4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:41.315135 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80512b26-efcd-4543-a068-33975ffc10e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:41.316499 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a58f2-5806-4ee0-8ec3-aab62e00b3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:41.317216 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a58f2-5806-4ee0-8ec3-aab62e00b3bd admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:41.317504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a58f2-5806-4ee0-8ec3-aab62e00b3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:41.317785 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a58f2-5806-4ee0-8ec3-aab62e00b3bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:41.318593 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a58f2-5806-4ee0-8ec3-aab62e00b3bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:41.320733 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a58f2-5806-4ee0-8ec3-aab62e00b3bd admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:41.363957 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-80512b26-efcd-4543-a068-33975ffc10e4 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:41.368330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-80512b26-efcd-4543-a068-33975ffc10e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:41.391298 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 962/1913] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:40 2022] POST /identity/v3/projects => generated 373 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:36:41.396112 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a58f2-5806-4ee0-8ec3-aab62e00b3bd admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:41.408340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-845a58f2-5806-4ee0-8ec3-aab62e00b3bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:41.411983 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 952/1914] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:41.413791 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ede1a6-d2d3-4d66-ba32-9e39af63d75f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:41.414233 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ede1a6-d2d3-4d66-ba32-9e39af63d75f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:41.415407 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ede1a6-d2d3-4d66-ba32-9e39af63d75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:41.465115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b5ede1a6-d2d3-4d66-ba32-9e39af63d75f None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:41.466965 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b5ede1a6-d2d3-4d66-ba32-9e39af63d75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:41.482150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:41.566537 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a58f2-5806-4ee0-8ec3-aab62e00b3bd admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:41.581356 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa65da3-7497-4be2-96b9-eb2615afd0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:41.582936 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa65da3-7497-4be2-96b9-eb2615afd0d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:41.584158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa65da3-7497-4be2-96b9-eb2615afd0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:41.585510 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa65da3-7497-4be2-96b9-eb2615afd0d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:41.587763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa65da3-7497-4be2-96b9-eb2615afd0d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:41.594129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa65da3-7497-4be2-96b9-eb2615afd0d0 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:41.644179 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 963/1915] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:41 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:41.647674 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa65da3-7497-4be2-96b9-eb2615afd0d0 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:41.651736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 953/1916] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:41.660114 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ede1a6-d2d3-4d66-ba32-9e39af63d75f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:41.673831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ede1a6-d2d3-4d66-ba32-9e39af63d75f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:41.677606 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa65da3-7497-4be2-96b9-eb2615afd0d0 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:41.688165 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cc13f-4a8d-44f6-ac98-799a0c66fa4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:41.689389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cc13f-4a8d-44f6-ac98-799a0c66fa4f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:41.689876 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cc13f-4a8d-44f6-ac98-799a0c66fa4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:41.690348 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2a316-66ed-4875-9b41-12f6d445bc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:41.690720 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cc13f-4a8d-44f6-ac98-799a0c66fa4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:41.691580 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2a316-66ed-4875-9b41-12f6d445bc7d admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:41.691931 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2a316-66ed-4875-9b41-12f6d445bc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:41.692287 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2a316-66ed-4875-9b41-12f6d445bc7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:41.692884 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cc13f-4a8d-44f6-ac98-799a0c66fa4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:36:41.693212 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cc13f-4a8d-44f6-ac98-799a0c66fa4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:41.694784 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2a316-66ed-4875-9b41-12f6d445bc7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:41.697811 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155cc13f-4a8d-44f6-ac98-799a0c66fa4f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:41.698799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2a316-66ed-4875-9b41-12f6d445bc7d admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:41.730296 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 954/1917] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:36:41.736475 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cc13f-4a8d-44f6-ac98-799a0c66fa4f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:41.740441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 964/1918] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:41.741589 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2a316-66ed-4875-9b41-12f6d445bc7d admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:41.761199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cc13f-4a8d-44f6-ac98-799a0c66fa4f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:41.767465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2a316-66ed-4875-9b41-12f6d445bc7d admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:41.776744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cf537-8279-428c-b6d7-d55af31188de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:41.778427 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cf537-8279-428c-b6d7-d55af31188de admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:41.779248 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cf537-8279-428c-b6d7-d55af31188de admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:41.779248 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cf537-8279-428c-b6d7-d55af31188de admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:41.782155 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289b291-0a1e-40a0-a3cb-f8414d40b921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:41.783384 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289b291-0a1e-40a0-a3cb-f8414d40b921 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:41.783672 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289b291-0a1e-40a0-a3cb-f8414d40b921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:41.784763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2cf537-8279-428c-b6d7-d55af31188de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:41.787742 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2cf537-8279-428c-b6d7-d55af31188de admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:41.788623 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289b291-0a1e-40a0-a3cb-f8414d40b921 admin admin] PATH_INFO: `/v3/projects/3b28f44d18ed4241b89d63e9e28e1c3a/users/ad37ce5c18194991b8b908ecf276a135/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:41.827392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:41.853017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289b291-0a1e-40a0-a3cb-f8414d40b921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b28f44d18ed4241b89d63e9e28e1c3a, user_id=ad37ce5c18194991b8b908ecf276a135, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:41.859695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289b291-0a1e-40a0-a3cb-f8414d40b921 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:41.884121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 955/1919] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:36:41 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:41.890125 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cf537-8279-428c-b6d7-d55af31188de admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:41.919574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 965/1920] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:41 2022] PUT /identity/v3/projects/3b28f44d18ed4241b89d63e9e28e1c3a/users/ad37ce5c18194991b8b908ecf276a135/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:41.924927 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289b291-0a1e-40a0-a3cb-f8414d40b921 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:42.044990 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cf537-8279-428c-b6d7-d55af31188de admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:42.096133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289b291-0a1e-40a0-a3cb-f8414d40b921 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:42.110460 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd72fb-7372-4f81-a74c-43486af25d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:42.111358 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd72fb-7372-4f81-a74c-43486af25d79 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:42.111649 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd72fb-7372-4f81-a74c-43486af25d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:42.111933 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd72fb-7372-4f81-a74c-43486af25d79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:42.113289 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd72fb-7372-4f81-a74c-43486af25d79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:42.114901 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd72fb-7372-4f81-a74c-43486af25d79 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:42.152108 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 966/1921] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:42.154677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd72fb-7372-4f81-a74c-43486af25d79 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:42.164906 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9769b856-fbd9-429c-a389-91fcf02a5718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:42.167036 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769b856-fbd9-429c-a389-91fcf02a5718 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:42.167417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769b856-fbd9-429c-a389-91fcf02a5718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:42.168070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769b856-fbd9-429c-a389-91fcf02a5718 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:42.169822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769b856-fbd9-429c-a389-91fcf02a5718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:42.173370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9769b856-fbd9-429c-a389-91fcf02a5718 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:42.174556 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd72fb-7372-4f81-a74c-43486af25d79 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:42.183795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efeb439-bcbb-4815-8f6e-b15e2e6f27a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:42.184926 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efeb439-bcbb-4815-8f6e-b15e2e6f27a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:42.185217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efeb439-bcbb-4815-8f6e-b15e2e6f27a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:42.185501 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efeb439-bcbb-4815-8f6e-b15e2e6f27a3 admin admin] PATH_INFO: `/v3/projects/3b28f44d18ed4241b89d63e9e28e1c3a/users/ad37ce5c18194991b8b908ecf276a135/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:42.188206 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efeb439-bcbb-4815-8f6e-b15e2e6f27a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b28f44d18ed4241b89d63e9e28e1c3a, user_id=ad37ce5c18194991b8b908ecf276a135, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:42.191508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efeb439-bcbb-4815-8f6e-b15e2e6f27a3 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:42.212156 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 967/1922] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:42 2022] PUT /identity/v3/projects/3b28f44d18ed4241b89d63e9e28e1c3a/users/ad37ce5c18194991b8b908ecf276a135/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:36:42.216338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 956/1923] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:36:41 2022] POST /identity/v3/users => generated 288 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:36:42.223451 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efeb439-bcbb-4815-8f6e-b15e2e6f27a3 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:42.224881 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7264da-92f9-4baf-9b65-14652d1def4a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:42.226085 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7264da-92f9-4baf-9b65-14652d1def4a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:42.228111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7264da-92f9-4baf-9b65-14652d1def4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:42.280806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:42.296113 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ec7264da-92f9-4baf-9b65-14652d1def4a None None] MFA Rules not processed for user `ad37ce5c18194991b8b908ecf276a135`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:42.297374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ec7264da-92f9-4baf-9b65-14652d1def4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:42.367426 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efeb439-bcbb-4815-8f6e-b15e2e6f27a3 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:42.393088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39d834-92e0-4a2b-990d-bbb6d9f57d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:42.394311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39d834-92e0-4a2b-990d-bbb6d9f57d58 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:42.394754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39d834-92e0-4a2b-990d-bbb6d9f57d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:42.395308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e39d834-92e0-4a2b-990d-bbb6d9f57d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:42.396232 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39d834-92e0-4a2b-990d-bbb6d9f57d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:42.399732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e39d834-92e0-4a2b-990d-bbb6d9f57d58 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:42.424297 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 968/1924] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:42.429824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39d834-92e0-4a2b-990d-bbb6d9f57d58 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:42.451251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e39d834-92e0-4a2b-990d-bbb6d9f57d58 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:42.458015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e942331-d7b3-4235-be6d-ec37eef49b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:42.458730 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e942331-d7b3-4235-be6d-ec37eef49b61 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:42.459084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e942331-d7b3-4235-be6d-ec37eef49b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:42.459374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e942331-d7b3-4235-be6d-ec37eef49b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:42.460113 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e942331-d7b3-4235-be6d-ec37eef49b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:42.461722 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e942331-d7b3-4235-be6d-ec37eef49b61 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:42.474583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 957/1925] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:36:42.488195 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7264da-92f9-4baf-9b65-14652d1def4a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:42.490093 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ec7264da-92f9-4baf-9b65-14652d1def4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:42.491069 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 969/1926] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:36:42.492061 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e942331-d7b3-4235-be6d-ec37eef49b61 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:42.507414 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e942331-d7b3-4235-be6d-ec37eef49b61 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:42.513398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e442f3b-c045-4309-a455-bfbd05107da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:42.514372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e442f3b-c045-4309-a455-bfbd05107da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:42.514810 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e442f3b-c045-4309-a455-bfbd05107da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:42.515139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e442f3b-c045-4309-a455-bfbd05107da3 admin admin] PATH_INFO: `/v3/projects/2cbfb944c43d4411aaa3c9bae1b2cce3/users/a87f2662884542658bb0aa314edd79fd/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:42.538170 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e442f3b-c045-4309-a455-bfbd05107da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cbfb944c43d4411aaa3c9bae1b2cce3, user_id=a87f2662884542658bb0aa314edd79fd, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:42.541447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e442f3b-c045-4309-a455-bfbd05107da3 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:42.558313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 970/1927] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:36:42 2022] PUT /identity/v3/projects/2cbfb944c43d4411aaa3c9bae1b2cce3/users/a87f2662884542658bb0aa314edd79fd/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:36:42.565262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7264da-92f9-4baf-9b65-14652d1def4a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:42.570688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d2090-60f6-481c-bf39-43abe833a5d6 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:42.571048 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d2090-60f6-481c-bf39-43abe833a5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:42.571334 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d2090-60f6-481c-bf39-43abe833a5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:42.602514 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-7b1d2090-60f6-481c-bf39-43abe833a5d6 None None] MFA Rules not processed for user `a87f2662884542658bb0aa314edd79fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:36:42.604603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7b1d2090-60f6-481c-bf39-43abe833a5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:42.629766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:42.640648 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a2593-207e-4afd-b6b6-024f630391d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:42.641568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a2593-207e-4afd-b6b6-024f630391d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:42.642029 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a2593-207e-4afd-b6b6-024f630391d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:42.642483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a2593-207e-4afd-b6b6-024f630391d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:42.692480 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 971/1928] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:36:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:36:42.704757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d2090-60f6-481c-bf39-43abe833a5d6 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:42.706190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7b1d2090-60f6-481c-bf39-43abe833a5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:42.718627 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a2593-207e-4afd-b6b6-024f630391d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:42.719158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d2090-60f6-481c-bf39-43abe833a5d6 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:42.720120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a2593-207e-4afd-b6b6-024f630391d9 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:42.723037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f2a40-1142-43b9-8d59-e7fee7c63c63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:42.723734 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f2a40-1142-43b9-8d59-e7fee7c63c63 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:42.724024 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f2a40-1142-43b9-8d59-e7fee7c63c63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:42.724437 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f2a40-1142-43b9-8d59-e7fee7c63c63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:42.731256 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 958/1929] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:42.737322 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f2a40-1142-43b9-8d59-e7fee7c63c63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:42.738762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f2a40-1142-43b9-8d59-e7fee7c63c63 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:42.748228 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 972/1930] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:36:42.806333 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a2593-207e-4afd-b6b6-024f630391d9 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:36:42.811232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:42.817225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a2593-207e-4afd-b6b6-024f630391d9 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:36:42.832415 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099b23d-e409-4e4d-83bc-761b658f23db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:36:42.833324 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099b23d-e409-4e4d-83bc-761b658f23db service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:36:42.833804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099b23d-e409-4e4d-83bc-761b658f23db service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:36:42.834198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099b23d-e409-4e4d-83bc-761b658f23db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:36:42.836357 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7099b23d-e409-4e4d-83bc-761b658f23db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:36:42.916170 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099b23d-e409-4e4d-83bc-761b658f23db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:36:42.917736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099b23d-e409-4e4d-83bc-761b658f23db service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:36:42.932670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 959/1931] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:36:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:36:52.743665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=23 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:36:52.925196 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=27 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:05.149268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f2a40-1142-43b9-8d59-e7fee7c63c63 service cinder] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:05.152298 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b98f2a40-1142-43b9-8d59-e7fee7c63c63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:05.222275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099b23d-e409-4e4d-83bc-761b658f23db service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:05.223867 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7099b23d-e409-4e4d-83bc-761b658f23db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:05.265257 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f2a40-1142-43b9-8d59-e7fee7c63c63 service cinder] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:05.269853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ec273-44e4-4837-bc07-a6fbd4f02805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:05.270688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ec273-44e4-4837-bc07-a6fbd4f02805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:05.271009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ec273-44e4-4837-bc07-a6fbd4f02805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:05.271290 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ec273-44e4-4837-bc07-a6fbd4f02805 admin admin] PATH_INFO: `/v3/users/8eadb70438a2414a8aa5197c78b3b272` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:05.272492 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ec273-44e4-4837-bc07-a6fbd4f02805 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eadb70438a2414a8aa5197c78b3b272)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:05.274451 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ec273-44e4-4837-bc07-a6fbd4f02805 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:05.300260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099b23d-e409-4e4d-83bc-761b658f23db service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:05.305654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb73e2f-b932-486f-8033-14be00f2ef98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:05.307118 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb73e2f-b932-486f-8033-14be00f2ef98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:05.307394 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb73e2f-b932-486f-8033-14be00f2ef98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:05.307677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb73e2f-b932-486f-8033-14be00f2ef98 admin admin] PATH_INFO: `/v3/users/06245c8e10d947af87088187487762f6` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:05.309212 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb73e2f-b932-486f-8033-14be00f2ef98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06245c8e10d947af87088187487762f6)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:05.311480 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb73e2f-b932-486f-8033-14be00f2ef98 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:05.329551 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0d4ec273-44e4-4837-bc07-a6fbd4f02805 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eadb70438a2414a8aa5197c78b3b272'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:05.343889 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0d4ec273-44e4-4837-bc07-a6fbd4f02805 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eadb70438a2414a8aa5197c78b3b272'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:05.352075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0d4ec273-44e4-4837-bc07-a6fbd4f02805 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eadb70438a2414a8aa5197c78b3b272'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:05.360515 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0d4ec273-44e4-4837-bc07-a6fbd4f02805 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eadb70438a2414a8aa5197c78b3b272'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:05.361205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0d4ec273-44e4-4837-bc07-a6fbd4f02805 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eadb70438a2414a8aa5197c78b3b272'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:05.365158 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4eb73e2f-b932-486f-8033-14be00f2ef98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06245c8e10d947af87088187487762f6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:05.368394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 973/1932] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:37:05 2022] DELETE /identity/v3/users/8eadb70438a2414a8aa5197c78b3b272 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:05.376452 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe26829-a460-4001-8003-83f4842d53e3 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:05.376754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe26829-a460-4001-8003-83f4842d53e3 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:05.377042 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe26829-a460-4001-8003-83f4842d53e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:05.379896 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4eb73e2f-b932-486f-8033-14be00f2ef98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06245c8e10d947af87088187487762f6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:05.403761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4eb73e2f-b932-486f-8033-14be00f2ef98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06245c8e10d947af87088187487762f6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:05.404967 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4eb73e2f-b932-486f-8033-14be00f2ef98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06245c8e10d947af87088187487762f6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:05.405577 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-dbe26829-a460-4001-8003-83f4842d53e3 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:05.405895 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4eb73e2f-b932-486f-8033-14be00f2ef98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06245c8e10d947af87088187487762f6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:05.407536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-dbe26829-a460-4001-8003-83f4842d53e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:05.439380 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 960/1933] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:37:05 2022] DELETE /identity/v3/users/06245c8e10d947af87088187487762f6 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:37:05.492660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 974/1934] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:37:05.503200 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb73e2f-b932-486f-8033-14be00f2ef98 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:05.505496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4eb73e2f-b932-486f-8033-14be00f2ef98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:05.601319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb73e2f-b932-486f-8033-14be00f2ef98 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:05.614229 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278eaa8-a022-45a2-8761-1be99d43c1c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:05.614941 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278eaa8-a022-45a2-8761-1be99d43c1c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:05.615334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278eaa8-a022-45a2-8761-1be99d43c1c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:05.615622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278eaa8-a022-45a2-8761-1be99d43c1c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:05.691666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:05.756592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278eaa8-a022-45a2-8761-1be99d43c1c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:05.758423 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278eaa8-a022-45a2-8761-1be99d43c1c6 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:05.771445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 961/1935] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:05.811766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe26829-a460-4001-8003-83f4842d53e3 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:05.814302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-dbe26829-a460-4001-8003-83f4842d53e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:05.830019 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe26829-a460-4001-8003-83f4842d53e3 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:05.836995 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f0661-577d-4c21-8047-04016fdc32ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:05.837465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f0661-577d-4c21-8047-04016fdc32ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:05.837878 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f0661-577d-4c21-8047-04016fdc32ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:05.839194 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f0661-577d-4c21-8047-04016fdc32ac admin admin] PATH_INFO: `/v3/projects/68146ddf17d146a29ad9a23581e0ca6e` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:05.841697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f0661-577d-4c21-8047-04016fdc32ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=68146ddf17d146a29ad9a23581e0ca6e)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:05.844118 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f0661-577d-4c21-8047-04016fdc32ac admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:05.868611 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:05.932603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-514f0661-577d-4c21-8047-04016fdc32ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68146ddf17d146a29ad9a23581e0ca6e', 'request_id': 'req-514f0661-577d-4c21-8047-04016fdc32ac'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:05.942750 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-514f0661-577d-4c21-8047-04016fdc32ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68146ddf17d146a29ad9a23581e0ca6e', 'request_id': 'req-514f0661-577d-4c21-8047-04016fdc32ac'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:05.951203 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-514f0661-577d-4c21-8047-04016fdc32ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68146ddf17d146a29ad9a23581e0ca6e', 'request_id': 'req-514f0661-577d-4c21-8047-04016fdc32ac'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:05.966784 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-514f0661-577d-4c21-8047-04016fdc32ac admin admin] The token cache is being invalidate because project 68146ddf17d146a29ad9a23581e0ca6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:37:05.967271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-514f0661-577d-4c21-8047-04016fdc32ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:05.967980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-514f0661-577d-4c21-8047-04016fdc32ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:05.969669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 975/1936] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:37:05 2022] DELETE /identity/v3/projects/68146ddf17d146a29ad9a23581e0ca6e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:05.973936 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278eaa8-a022-45a2-8761-1be99d43c1c6 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:05.988850 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278eaa8-a022-45a2-8761-1be99d43c1c6 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:06.094425 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed6799-fedd-4329-b812-0f12a006db96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:06.095448 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed6799-fedd-4329-b812-0f12a006db96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:06.095759 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed6799-fedd-4329-b812-0f12a006db96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:06.096022 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed6799-fedd-4329-b812-0f12a006db96 admin admin] PATH_INFO: `/v3/users/266a80bbc9934ed295bd6e09c3b86e7f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:06.097746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed6799-fedd-4329-b812-0f12a006db96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=266a80bbc9934ed295bd6e09c3b86e7f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:06.099969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed6799-fedd-4329-b812-0f12a006db96 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:06.181242 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-26ed6799-fedd-4329-b812-0f12a006db96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '266a80bbc9934ed295bd6e09c3b86e7f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.191309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-26ed6799-fedd-4329-b812-0f12a006db96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '266a80bbc9934ed295bd6e09c3b86e7f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.201254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-26ed6799-fedd-4329-b812-0f12a006db96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '266a80bbc9934ed295bd6e09c3b86e7f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.201979 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-26ed6799-fedd-4329-b812-0f12a006db96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '266a80bbc9934ed295bd6e09c3b86e7f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.202645 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-26ed6799-fedd-4329-b812-0f12a006db96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '266a80bbc9934ed295bd6e09c3b86e7f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.219172 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f0661-577d-4c21-8047-04016fdc32ac admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:06.221543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-514f0661-577d-4c21-8047-04016fdc32ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:06.234807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 962/1937] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:37:05 2022] DELETE /identity/v3/users/266a80bbc9934ed295bd6e09c3b86e7f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:06.245896 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f0661-577d-4c21-8047-04016fdc32ac admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:06.341737 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35984bcc-d848-4480-954b-ac1dd4e7a2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:06.344857 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35984bcc-d848-4480-954b-ac1dd4e7a2a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:06.344857 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35984bcc-d848-4480-954b-ac1dd4e7a2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:06.344857 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35984bcc-d848-4480-954b-ac1dd4e7a2a4 admin admin] PATH_INFO: `/v3/projects/0ea6f040d7b14d3bb45fdb45190692ba` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:06.344857 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35984bcc-d848-4480-954b-ac1dd4e7a2a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ea6f040d7b14d3bb45fdb45190692ba)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:06.346361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35984bcc-d848-4480-954b-ac1dd4e7a2a4 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:06.394572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed6799-fedd-4329-b812-0f12a006db96 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:06.396586 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-26ed6799-fedd-4329-b812-0f12a006db96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:06.406890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-35984bcc-d848-4480-954b-ac1dd4e7a2a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ea6f040d7b14d3bb45fdb45190692ba', 'request_id': 'req-35984bcc-d848-4480-954b-ac1dd4e7a2a4'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.411966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed6799-fedd-4329-b812-0f12a006db96 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:06.415797 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-35984bcc-d848-4480-954b-ac1dd4e7a2a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ea6f040d7b14d3bb45fdb45190692ba', 'request_id': 'req-35984bcc-d848-4480-954b-ac1dd4e7a2a4'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.426406 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-35984bcc-d848-4480-954b-ac1dd4e7a2a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ea6f040d7b14d3bb45fdb45190692ba', 'request_id': 'req-35984bcc-d848-4480-954b-ac1dd4e7a2a4'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.438148 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-35984bcc-d848-4480-954b-ac1dd4e7a2a4 admin admin] The token cache is being invalidate because project 0ea6f040d7b14d3bb45fdb45190692ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:37:06.438574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-35984bcc-d848-4480-954b-ac1dd4e7a2a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.439353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-35984bcc-d848-4480-954b-ac1dd4e7a2a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.444295 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 976/1938] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:37:06 2022] DELETE /identity/v3/projects/0ea6f040d7b14d3bb45fdb45190692ba => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:37:06.445825 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3fdba-45d2-4ba9-937b-c62c42ae7c2a None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:06.445991 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3fdba-45d2-4ba9-937b-c62c42ae7c2a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:06.446717 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3fdba-45d2-4ba9-937b-c62c42ae7c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:06.471493 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-ddb3fdba-45d2-4ba9-937b-c62c42ae7c2a None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:06.471622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:06.473110 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-ddb3fdba-45d2-4ba9-937b-c62c42ae7c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:06.487032 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fb081-20d1-4b71-86b7-6e78145acc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:06.492522 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fb081-20d1-4b71-86b7-6e78145acc7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:06.493205 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fb081-20d1-4b71-86b7-6e78145acc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:06.493501 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fb081-20d1-4b71-86b7-6e78145acc7a admin admin] PATH_INFO: `/v3/users/a87f2662884542658bb0aa314edd79fd` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:06.494915 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fb081-20d1-4b71-86b7-6e78145acc7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a87f2662884542658bb0aa314edd79fd)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:06.498750 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fb081-20d1-4b71-86b7-6e78145acc7a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:06.520876 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:06.568326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 977/1939] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:06.569377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3fdba-45d2-4ba9-937b-c62c42ae7c2a None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:06.571322 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-ddb3fdba-45d2-4ba9-937b-c62c42ae7c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:06.577980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-590fb081-20d1-4b71-86b7-6e78145acc7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a87f2662884542658bb0aa314edd79fd'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.583419 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3fdba-45d2-4ba9-937b-c62c42ae7c2a None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:06.586826 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-590fb081-20d1-4b71-86b7-6e78145acc7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a87f2662884542658bb0aa314edd79fd'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.588030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d072f-31dc-4b0d-a527-b30a37894817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:06.588889 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d072f-31dc-4b0d-a527-b30a37894817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:06.589173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d072f-31dc-4b0d-a527-b30a37894817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:06.589948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d072f-31dc-4b0d-a527-b30a37894817 admin admin] PATH_INFO: `/v3/projects/2811546d503e474bbdfe365e6d67a53c` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:06.592069 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d072f-31dc-4b0d-a527-b30a37894817 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2811546d503e474bbdfe365e6d67a53c)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:06.594599 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d072f-31dc-4b0d-a527-b30a37894817 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:06.596670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-590fb081-20d1-4b71-86b7-6e78145acc7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a87f2662884542658bb0aa314edd79fd'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.597554 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-590fb081-20d1-4b71-86b7-6e78145acc7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a87f2662884542658bb0aa314edd79fd'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.598408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-590fb081-20d1-4b71-86b7-6e78145acc7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a87f2662884542658bb0aa314edd79fd'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.621408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 963/1940] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:37:06 2022] DELETE /identity/v3/users/a87f2662884542658bb0aa314edd79fd => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:06.623143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867f0dd-a43a-4afb-a7b3-2a5a9873b55b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:06.623440 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867f0dd-a43a-4afb-a7b3-2a5a9873b55b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:06.623726 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867f0dd-a43a-4afb-a7b3-2a5a9873b55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:06.643166 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8867f0dd-a43a-4afb-a7b3-2a5a9873b55b None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:06.644399 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8867f0dd-a43a-4afb-a7b3-2a5a9873b55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:06.650745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-a41d072f-31dc-4b0d-a527-b30a37894817 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2811546d503e474bbdfe365e6d67a53c', 'request_id': 'req-a41d072f-31dc-4b0d-a527-b30a37894817'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.659316 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-a41d072f-31dc-4b0d-a527-b30a37894817 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2811546d503e474bbdfe365e6d67a53c', 'request_id': 'req-a41d072f-31dc-4b0d-a527-b30a37894817'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.668936 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-a41d072f-31dc-4b0d-a527-b30a37894817 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2811546d503e474bbdfe365e6d67a53c', 'request_id': 'req-a41d072f-31dc-4b0d-a527-b30a37894817'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.680416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-a41d072f-31dc-4b0d-a527-b30a37894817 admin admin] The token cache is being invalidate because project 2811546d503e474bbdfe365e6d67a53c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:37:06.680765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-a41d072f-31dc-4b0d-a527-b30a37894817 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.681591 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-a41d072f-31dc-4b0d-a527-b30a37894817 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:06.684026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 978/1941] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:37:06 2022] DELETE /identity/v3/projects/2811546d503e474bbdfe365e6d67a53c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:06.685636 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487c9ff-b5a1-4fe8-b729-e8e800352cba None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:06.685928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487c9ff-b5a1-4fe8-b729-e8e800352cba None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:06.686215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487c9ff-b5a1-4fe8-b729-e8e800352cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:06.704793 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-3487c9ff-b5a1-4fe8-b729-e8e800352cba None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:06.706459 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3487c9ff-b5a1-4fe8-b729-e8e800352cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:06.712285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 964/1942] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:06.713565 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557e5cc-1efb-4c2c-9eab-b56e5c819a1a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:06.713866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557e5cc-1efb-4c2c-9eab-b56e5c819a1a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:06.714153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557e5cc-1efb-4c2c-9eab-b56e5c819a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:06.718330 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 979/1943] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:06.719484 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487c9ff-b5a1-4fe8-b729-e8e800352cba None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:06.732881 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487c9ff-b5a1-4fe8-b729-e8e800352cba None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:06.737838 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d557e5cc-1efb-4c2c-9eab-b56e5c819a1a None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:06.738155 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614cb0cf-c9d0-4b31-9417-2e57c06d0534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:06.739092 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d557e5cc-1efb-4c2c-9eab-b56e5c819a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:06.739215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cb0cf-c9d0-4b31-9417-2e57c06d0534 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:06.739636 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cb0cf-c9d0-4b31-9417-2e57c06d0534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:06.740045 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614cb0cf-c9d0-4b31-9417-2e57c06d0534 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:06.741122 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cb0cf-c9d0-4b31-9417-2e57c06d0534 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:37:06.741440 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cb0cf-c9d0-4b31-9417-2e57c06d0534 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:06.743204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614cb0cf-c9d0-4b31-9417-2e57c06d0534 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:06.748044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 965/1944] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:06.748895 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557e5cc-1efb-4c2c-9eab-b56e5c819a1a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:06.750187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d557e5cc-1efb-4c2c-9eab-b56e5c819a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:06.761135 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 980/1945] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:37:06.762812 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d17f9-3fbb-418a-a905-d41e5acaf9db None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:06.763484 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d17f9-3fbb-418a-a905-d41e5acaf9db None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:06.763484 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d17f9-3fbb-418a-a905-d41e5acaf9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:06.783307 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-974d17f9-3fbb-418a-a905-d41e5acaf9db None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:06.784797 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-974d17f9-3fbb-418a-a905-d41e5acaf9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:06.793606 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 981/1946] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:06.794947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d17f9-3fbb-418a-a905-d41e5acaf9db None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:06.804870 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d17f9-3fbb-418a-a905-d41e5acaf9db None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:06.813360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c3ed8-fae5-4526-8854-38c1e19b226b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:06.813360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c3ed8-fae5-4526-8854-38c1e19b226b admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:06.813360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c3ed8-fae5-4526-8854-38c1e19b226b admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:06.813360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c3ed8-fae5-4526-8854-38c1e19b226b admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:06.813769 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c3ed8-fae5-4526-8854-38c1e19b226b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:06.813868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557e5cc-1efb-4c2c-9eab-b56e5c819a1a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:06.813958 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c3ed8-fae5-4526-8854-38c1e19b226b admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:06.823492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0a143-0ba3-4ba7-a79a-94c82d48a5fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:06.824153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0a143-0ba3-4ba7-a79a-94c82d48a5fc service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:06.824446 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0a143-0ba3-4ba7-a79a-94c82d48a5fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:06.824740 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf0a143-0ba3-4ba7-a79a-94c82d48a5fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:06.835153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf0a143-0ba3-4ba7-a79a-94c82d48a5fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:06.836525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf0a143-0ba3-4ba7-a79a-94c82d48a5fc service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:06.848281 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 966/1947] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:06.848977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0a143-0ba3-4ba7-a79a-94c82d48a5fc service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:06.851400 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 982/1948] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:37:06 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:06.855256 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c3ed8-fae5-4526-8854-38c1e19b226b admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:06.859482 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:06.866286 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf0a143-0ba3-4ba7-a79a-94c82d48a5fc service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:06.936292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c3ed8-fae5-4526-8854-38c1e19b226b admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:06.943324 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e95c96-b602-498b-87d7-07d8d87f2bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:06.944509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e95c96-b602-498b-87d7-07d8d87f2bbe admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:06.945079 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e95c96-b602-498b-87d7-07d8d87f2bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:06.945583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e95c96-b602-498b-87d7-07d8d87f2bbe admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:06.946633 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e95c96-b602-498b-87d7-07d8d87f2bbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:06.949558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e95c96-b602-498b-87d7-07d8d87f2bbe admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:06.954271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09801243-eef2-4dad-86f5-108610ecb6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:06.955294 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09801243-eef2-4dad-86f5-108610ecb6a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:06.955592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09801243-eef2-4dad-86f5-108610ecb6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:06.956013 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09801243-eef2-4dad-86f5-108610ecb6a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:06.956769 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09801243-eef2-4dad-86f5-108610ecb6a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:37:06.957088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09801243-eef2-4dad-86f5-108610ecb6a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:06.958877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09801243-eef2-4dad-86f5-108610ecb6a7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:06.974965 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:06.987453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 967/1949] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:37:06.988476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 983/1950] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:37:06 2022] POST /identity/v3/users => generated 287 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:06.991712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09801243-eef2-4dad-86f5-108610ecb6a7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:06.993052 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e95c96-b602-498b-87d7-07d8d87f2bbe admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:07.008319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09801243-eef2-4dad-86f5-108610ecb6a7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:07.014622 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e95c96-b602-498b-87d7-07d8d87f2bbe admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:07.014807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4aa4a2-05ed-4615-9dcc-77fd82779ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:07.016113 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4aa4a2-05ed-4615-9dcc-77fd82779ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:07.016113 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4aa4a2-05ed-4615-9dcc-77fd82779ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:07.016521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4aa4a2-05ed-4615-9dcc-77fd82779ae1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:07.017511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4aa4a2-05ed-4615-9dcc-77fd82779ae1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:07.020366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4aa4a2-05ed-4615-9dcc-77fd82779ae1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:07.029071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160af6d0-0f23-427f-b675-407efb8d3007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:07.029781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160af6d0-0f23-427f-b675-407efb8d3007 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:07.030077 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160af6d0-0f23-427f-b675-407efb8d3007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:07.030367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160af6d0-0f23-427f-b675-407efb8d3007 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:07.031200 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160af6d0-0f23-427f-b675-407efb8d3007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:07.033035 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160af6d0-0f23-427f-b675-407efb8d3007 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:07.052924 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 984/1951] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:07.057628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160af6d0-0f23-427f-b675-407efb8d3007 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:07.073110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 968/1952] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:37:06 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:37:07.077615 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4aa4a2-05ed-4615-9dcc-77fd82779ae1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:07.078731 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160af6d0-0f23-427f-b675-407efb8d3007 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:07.183576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e970ce6-b008-407b-be81-248d6241e8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:07.184699 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e970ce6-b008-407b-be81-248d6241e8dc admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:07.185183 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e970ce6-b008-407b-be81-248d6241e8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:07.186631 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e970ce6-b008-407b-be81-248d6241e8dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:07.188632 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e970ce6-b008-407b-be81-248d6241e8dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:07.191916 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e970ce6-b008-407b-be81-248d6241e8dc admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:07.209813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4aa4a2-05ed-4615-9dcc-77fd82779ae1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:07.228889 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52692eeb-7411-42ad-9b1e-0175ff924678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:07.228889 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52692eeb-7411-42ad-9b1e-0175ff924678 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:07.228889 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52692eeb-7411-42ad-9b1e-0175ff924678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:07.228889 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52692eeb-7411-42ad-9b1e-0175ff924678 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:07.229526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 985/1953] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:07.230183 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52692eeb-7411-42ad-9b1e-0175ff924678 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:07.230499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52692eeb-7411-42ad-9b1e-0175ff924678 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:07.231607 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e970ce6-b008-407b-be81-248d6241e8dc admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:07.256093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e970ce6-b008-407b-be81-248d6241e8dc admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:07.260553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 969/1954] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:37:07 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:07.261321 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb601337-3f4a-464a-8eed-bcb1c6e99725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:07.267132 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb601337-3f4a-464a-8eed-bcb1c6e99725 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:07.267132 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb601337-3f4a-464a-8eed-bcb1c6e99725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:07.267132 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb601337-3f4a-464a-8eed-bcb1c6e99725 admin admin] PATH_INFO: `/v3/projects/c4a9185149e24188b3bc35dadc176db4/users/6c8f66937d424eeeb6e83a0394d6df25/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:07.267381 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52692eeb-7411-42ad-9b1e-0175ff924678 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:07.278004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52692eeb-7411-42ad-9b1e-0175ff924678 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:07.280330 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb601337-3f4a-464a-8eed-bcb1c6e99725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4a9185149e24188b3bc35dadc176db4, user_id=6c8f66937d424eeeb6e83a0394d6df25, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:07.283118 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb601337-3f4a-464a-8eed-bcb1c6e99725 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:07.285848 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc0a2a-c521-44db-ab81-eb7c1d89d837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:07.287226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc0a2a-c521-44db-ab81-eb7c1d89d837 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:07.288024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc0a2a-c521-44db-ab81-eb7c1d89d837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:07.288665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc0a2a-c521-44db-ab81-eb7c1d89d837 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:07.290626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc0a2a-c521-44db-ab81-eb7c1d89d837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:07.292566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc0a2a-c521-44db-ab81-eb7c1d89d837 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:07.306645 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 970/1955] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:07.315598 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc0a2a-c521-44db-ab81-eb7c1d89d837 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:07.326968 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 986/1956] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:07 2022] PUT /identity/v3/projects/c4a9185149e24188b3bc35dadc176db4/users/6c8f66937d424eeeb6e83a0394d6df25/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:07.335961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc0a2a-c521-44db-ab81-eb7c1d89d837 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:07.340570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73a1fa-9102-4288-b73c-40949a13c3d2 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:07.340872 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73a1fa-9102-4288-b73c-40949a13c3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:07.341169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73a1fa-9102-4288-b73c-40949a13c3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:07.385736 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-fc73a1fa-9102-4288-b73c-40949a13c3d2 None None] MFA Rules not processed for user `6c8f66937d424eeeb6e83a0394d6df25`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:07.388447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-fc73a1fa-9102-4288-b73c-40949a13c3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:07.415284 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:07.428191 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789d34c-e316-48b1-89c3-4e9202a09bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:07.429928 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789d34c-e316-48b1-89c3-4e9202a09bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:07.430362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789d34c-e316-48b1-89c3-4e9202a09bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:07.430760 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789d34c-e316-48b1-89c3-4e9202a09bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:07.431857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789d34c-e316-48b1-89c3-4e9202a09bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:07.434966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789d34c-e316-48b1-89c3-4e9202a09bc7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:07.449566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 971/1957] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:37:07.450469 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789d34c-e316-48b1-89c3-4e9202a09bc7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:07.452387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2789d34c-e316-48b1-89c3-4e9202a09bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:07.469106 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789d34c-e316-48b1-89c3-4e9202a09bc7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:07.474484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e532f-59b2-48c9-8fe9-01730f88e0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:07.475416 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e532f-59b2-48c9-8fe9-01730f88e0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:07.475704 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e532f-59b2-48c9-8fe9-01730f88e0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:07.475986 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e532f-59b2-48c9-8fe9-01730f88e0f0 admin admin] PATH_INFO: `/v3/projects/2cbfb944c43d4411aaa3c9bae1b2cce3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:07.477209 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e532f-59b2-48c9-8fe9-01730f88e0f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cbfb944c43d4411aaa3c9bae1b2cce3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:07.664881 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 987/1958] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:37:07.664976 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e532f-59b2-48c9-8fe9-01730f88e0f0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:07.665150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73a1fa-9102-4288-b73c-40949a13c3d2 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:07.665150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73a1fa-9102-4288-b73c-40949a13c3d2 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:07.665150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01f022-386d-49d4-8241-a1ebb6ce49d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:07.665150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01f022-386d-49d4-8241-a1ebb6ce49d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:07.665631 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01f022-386d-49d4-8241-a1ebb6ce49d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:07.665631 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01f022-386d-49d4-8241-a1ebb6ce49d7 admin admin] PATH_INFO: `/v3/projects/e4fb6c541ee04ebea81001c24c67d779/users/bbe630f5c7c243588c5bd33023775d69/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:07.665631 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01f022-386d-49d4-8241-a1ebb6ce49d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4fb6c541ee04ebea81001c24c67d779, user_id=bbe630f5c7c243588c5bd33023775d69, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:07.665631 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01f022-386d-49d4-8241-a1ebb6ce49d7 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:07.677691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 988/1959] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:07 2022] PUT /identity/v3/projects/e4fb6c541ee04ebea81001c24c67d779/users/bbe630f5c7c243588c5bd33023775d69/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:07.678785 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01f022-386d-49d4-8241-a1ebb6ce49d7 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:07.680395 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-4c01f022-386d-49d4-8241-a1ebb6ce49d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:07.697525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-126e532f-59b2-48c9-8fe9-01730f88e0f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cbfb944c43d4411aaa3c9bae1b2cce3', 'request_id': 'req-126e532f-59b2-48c9-8fe9-01730f88e0f0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:07.708566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-126e532f-59b2-48c9-8fe9-01730f88e0f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cbfb944c43d4411aaa3c9bae1b2cce3', 'request_id': 'req-126e532f-59b2-48c9-8fe9-01730f88e0f0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:07.715477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:07.721656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-126e532f-59b2-48c9-8fe9-01730f88e0f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cbfb944c43d4411aaa3c9bae1b2cce3', 'request_id': 'req-126e532f-59b2-48c9-8fe9-01730f88e0f0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:07.739047 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-126e532f-59b2-48c9-8fe9-01730f88e0f0 admin admin] The token cache is being invalidate because project 2cbfb944c43d4411aaa3c9bae1b2cce3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:37:07.739426 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-126e532f-59b2-48c9-8fe9-01730f88e0f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:07.740360 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-126e532f-59b2-48c9-8fe9-01730f88e0f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:07.741994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 972/1960] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:37:07 2022] DELETE /identity/v3/projects/2cbfb944c43d4411aaa3c9bae1b2cce3 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:07.743890 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b09af-90b4-4fd3-9c3e-a6648fb838bc None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:07.743890 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b09af-90b4-4fd3-9c3e-a6648fb838bc None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:07.744120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b09af-90b4-4fd3-9c3e-a6648fb838bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:07.761288 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01f022-386d-49d4-8241-a1ebb6ce49d7 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:07.768008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c04b09af-90b4-4fd3-9c3e-a6648fb838bc None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:07.769319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c04b09af-90b4-4fd3-9c3e-a6648fb838bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:07.832149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5497887-71b7-4729-abff-692b12491b46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:07.832844 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5497887-71b7-4729-abff-692b12491b46 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:07.833107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5497887-71b7-4729-abff-692b12491b46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:07.833382 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5497887-71b7-4729-abff-692b12491b46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:07.842244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e5497887-71b7-4729-abff-692b12491b46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:07.883579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 973/1961] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:07.885005 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c437b-e0d7-4d02-97f6-db77bc7d2445 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:07.885386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c437b-e0d7-4d02-97f6-db77bc7d2445 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:07.885749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c437b-e0d7-4d02-97f6-db77bc7d2445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:07.907982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-982c437b-e0d7-4d02-97f6-db77bc7d2445 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:07.909776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-982c437b-e0d7-4d02-97f6-db77bc7d2445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:07.920614 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 974/1962] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:07.921368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c437b-e0d7-4d02-97f6-db77bc7d2445 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:07.935807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-982c437b-e0d7-4d02-97f6-db77bc7d2445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:07.943590 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5497887-71b7-4729-abff-692b12491b46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:07.945084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5497887-71b7-4729-abff-692b12491b46 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:07.955016 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 989/1963] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:07.956863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381924d-85fc-4d93-b782-579c6f6b4d4b None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:07.957553 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381924d-85fc-4d93-b782-579c6f6b4d4b None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:07.957939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381924d-85fc-4d93-b782-579c6f6b4d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:07.979189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-0381924d-85fc-4d93-b782-579c6f6b4d4b None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:07.980839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-0381924d-85fc-4d93-b782-579c6f6b4d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:07.990031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 990/1964] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:07.994086 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381924d-85fc-4d93-b782-579c6f6b4d4b None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:08.003555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c437b-e0d7-4d02-97f6-db77bc7d2445 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:08.004009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381924d-85fc-4d93-b782-579c6f6b4d4b None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:08.007797 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537fcf9-7efd-42e1-a002-48ee1b2df3ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:08.008637 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537fcf9-7efd-42e1-a002-48ee1b2df3ff service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:08.008888 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537fcf9-7efd-42e1-a002-48ee1b2df3ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:08.009400 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537fcf9-7efd-42e1-a002-48ee1b2df3ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:08.009669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae815b6-8d22-4fea-8a62-9351df10fb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:08.010491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae815b6-8d22-4fea-8a62-9351df10fb8a admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:08.010804 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae815b6-8d22-4fea-8a62-9351df10fb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:08.011245 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae815b6-8d22-4fea-8a62-9351df10fb8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:08.011985 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae815b6-8d22-4fea-8a62-9351df10fb8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:37:08.012384 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae815b6-8d22-4fea-8a62-9351df10fb8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:08.014271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae815b6-8d22-4fea-8a62-9351df10fb8a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:08.021340 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537fcf9-7efd-42e1-a002-48ee1b2df3ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:08.022785 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537fcf9-7efd-42e1-a002-48ee1b2df3ff service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:08.030684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 991/1965] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:37:08.033887 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae815b6-8d22-4fea-8a62-9351df10fb8a admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:08.038150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 975/1966] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:08.049115 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae815b6-8d22-4fea-8a62-9351df10fb8a admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:08.054432 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee6a60-51fb-4b22-8565-263c476ea3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:08.055230 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee6a60-51fb-4b22-8565-263c476ea3e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:08.055604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee6a60-51fb-4b22-8565-263c476ea3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:08.055960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee6a60-51fb-4b22-8565-263c476ea3e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:08.056724 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee6a60-51fb-4b22-8565-263c476ea3e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:08.059111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee6a60-51fb-4b22-8565-263c476ea3e1 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:08.093915 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 992/1967] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:37:08 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:37:08.098459 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537fcf9-7efd-42e1-a002-48ee1b2df3ff service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:08.110330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537fcf9-7efd-42e1-a002-48ee1b2df3ff service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:08.121032 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:08.205999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d06c5-7428-4c34-83ed-e9bab3e13b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:08.207029 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d06c5-7428-4c34-83ed-e9bab3e13b47 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:08.207452 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d06c5-7428-4c34-83ed-e9bab3e13b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:08.207910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d06c5-7428-4c34-83ed-e9bab3e13b47 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:08.209290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d06c5-7428-4c34-83ed-e9bab3e13b47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:08.212254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d06c5-7428-4c34-83ed-e9bab3e13b47 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:08.256247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee6a60-51fb-4b22-8565-263c476ea3e1 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:08.262774 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 976/1968] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:37:08 2022] POST /identity/v3/users => generated 283 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:08.271064 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee6a60-51fb-4b22-8565-263c476ea3e1 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:08.277022 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b55aa9-7d03-4911-a3f0-5cee8e2accff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:08.278642 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b55aa9-7d03-4911-a3f0-5cee8e2accff admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:08.279381 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b55aa9-7d03-4911-a3f0-5cee8e2accff admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:08.279917 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b55aa9-7d03-4911-a3f0-5cee8e2accff admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:08.282660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b55aa9-7d03-4911-a3f0-5cee8e2accff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:08.288596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b55aa9-7d03-4911-a3f0-5cee8e2accff admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:08.310798 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 993/1969] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:08.312645 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d06c5-7428-4c34-83ed-e9bab3e13b47 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:08.328675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d06c5-7428-4c34-83ed-e9bab3e13b47 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:08.334956 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f520b5a-7ad0-4586-8be3-b75443939dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:08.338765 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f520b5a-7ad0-4586-8be3-b75443939dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:08.339284 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f520b5a-7ad0-4586-8be3-b75443939dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:08.339660 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f520b5a-7ad0-4586-8be3-b75443939dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:08.340553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f520b5a-7ad0-4586-8be3-b75443939dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:08.342715 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f520b5a-7ad0-4586-8be3-b75443939dc0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:08.364871 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 977/1970] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:08 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:37:08.369136 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b55aa9-7d03-4911-a3f0-5cee8e2accff admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:08.382043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b55aa9-7d03-4911-a3f0-5cee8e2accff admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:08.393697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b587f-17f7-412f-a15a-9a62006bb13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:08.395464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b587f-17f7-412f-a15a-9a62006bb13a admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:08.396520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b587f-17f7-412f-a15a-9a62006bb13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:08.396993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b587f-17f7-412f-a15a-9a62006bb13a admin admin] PATH_INFO: `/v3/projects/3163d606f2a44729bc4233611cc83e03/users/11d39989ad86401c8665f2a63f6920c0/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:08.415716 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b587f-17f7-412f-a15a-9a62006bb13a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3163d606f2a44729bc4233611cc83e03, user_id=11d39989ad86401c8665f2a63f6920c0, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:08.418504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b587f-17f7-412f-a15a-9a62006bb13a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:08.434607 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 994/1971] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:08 2022] PUT /identity/v3/projects/3163d606f2a44729bc4233611cc83e03/users/11d39989ad86401c8665f2a63f6920c0/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:08.447210 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109fd8b-74b6-4e30-9b91-3c66d344498d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:08.448289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109fd8b-74b6-4e30-9b91-3c66d344498d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:08.448586 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4109fd8b-74b6-4e30-9b91-3c66d344498d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:08.480640 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4109fd8b-74b6-4e30-9b91-3c66d344498d None None] MFA Rules not processed for user `11d39989ad86401c8665f2a63f6920c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:08.481923 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4109fd8b-74b6-4e30-9b91-3c66d344498d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:08.573583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 978/1972] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:08.586464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b587f-17f7-412f-a15a-9a62006bb13a admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:08.588292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-562b587f-17f7-412f-a15a-9a62006bb13a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:08.620417 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:08.667815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b587f-17f7-412f-a15a-9a62006bb13a admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:08.672709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c215ed-0a49-479f-a8ba-f85c224feb9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:08.675477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c215ed-0a49-479f-a8ba-f85c224feb9d service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:08.675824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c215ed-0a49-479f-a8ba-f85c224feb9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:08.676160 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c215ed-0a49-479f-a8ba-f85c224feb9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:08.700705 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c215ed-0a49-479f-a8ba-f85c224feb9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:08.702288 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c215ed-0a49-479f-a8ba-f85c224feb9d service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:08.725434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 995/1973] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:08.927008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408a0f1-ec49-4cc4-adb5-942d985f298c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:08.927334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408a0f1-ec49-4cc4-adb5-942d985f298c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:08.927639 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408a0f1-ec49-4cc4-adb5-942d985f298c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:08.960482 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2408a0f1-ec49-4cc4-adb5-942d985f298c None None] MFA Rules not processed for user `bbe630f5c7c243588c5bd33023775d69`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:08.961847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2408a0f1-ec49-4cc4-adb5-942d985f298c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:09.073257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 979/1974] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:37:09.085501 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c215ed-0a49-479f-a8ba-f85c224feb9d service cinder] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:09.099959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c215ed-0a49-479f-a8ba-f85c224feb9d service cinder] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:09.104453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba94b0-6c70-4d52-8baf-ed60f188c800 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:09.105228 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba94b0-6c70-4d52-8baf-ed60f188c800 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:09.105617 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba94b0-6c70-4d52-8baf-ed60f188c800 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:09.105999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba94b0-6c70-4d52-8baf-ed60f188c800 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:09.119111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba94b0-6c70-4d52-8baf-ed60f188c800 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:09.120661 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba94b0-6c70-4d52-8baf-ed60f188c800 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:09.147680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 996/1975] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:10.588091 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e5054-76b7-4864-8aa3-80cb5ad75bb8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:10.588607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e5054-76b7-4864-8aa3-80cb5ad75bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:10.588710 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e5054-76b7-4864-8aa3-80cb5ad75bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:10.615039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-122e5054-76b7-4864-8aa3-80cb5ad75bb8 None None] MFA Rules not processed for user `bbe630f5c7c243588c5bd33023775d69`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:10.616511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-122e5054-76b7-4864-8aa3-80cb5ad75bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:10.625355 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 980/1976] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:11.248377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba94b0-6c70-4d52-8baf-ed60f188c800 service cinder] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:11.249975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-a2ba94b0-6c70-4d52-8baf-ed60f188c800 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:11.322090 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba94b0-6c70-4d52-8baf-ed60f188c800 service cinder] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:11.328518 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda90b1-4e01-416b-99c7-80c7654d9dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:11.329553 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda90b1-4e01-416b-99c7-80c7654d9dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:11.329769 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda90b1-4e01-416b-99c7-80c7654d9dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:11.329978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda90b1-4e01-416b-99c7-80c7654d9dfc admin admin] PATH_INFO: `/v3/users/6c8f66937d424eeeb6e83a0394d6df25` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:11.331355 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda90b1-4e01-416b-99c7-80c7654d9dfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c8f66937d424eeeb6e83a0394d6df25)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:11.334764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcda90b1-4e01-416b-99c7-80c7654d9dfc admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:11.397229 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-fcda90b1-4e01-416b-99c7-80c7654d9dfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c8f66937d424eeeb6e83a0394d6df25'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:11.412301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-fcda90b1-4e01-416b-99c7-80c7654d9dfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c8f66937d424eeeb6e83a0394d6df25'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:11.422874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-fcda90b1-4e01-416b-99c7-80c7654d9dfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c8f66937d424eeeb6e83a0394d6df25'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:11.430808 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-fcda90b1-4e01-416b-99c7-80c7654d9dfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c8f66937d424eeeb6e83a0394d6df25'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:11.431729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-fcda90b1-4e01-416b-99c7-80c7654d9dfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c8f66937d424eeeb6e83a0394d6df25'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:11.438651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 997/1977] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:37:11 2022] DELETE /identity/v3/users/6c8f66937d424eeeb6e83a0394d6df25 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:11.443422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8017961-6765-45aa-8062-f75f5de9a31f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:11.443721 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8017961-6765-45aa-8062-f75f5de9a31f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:11.444005 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8017961-6765-45aa-8062-f75f5de9a31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:11.449864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:11.466187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e8017961-6765-45aa-8062-f75f5de9a31f None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:11.467336 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e8017961-6765-45aa-8062-f75f5de9a31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:11.538499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 981/1978] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:37:11.549928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda90b1-4e01-416b-99c7-80c7654d9dfc admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:11.551652 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-fcda90b1-4e01-416b-99c7-80c7654d9dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:11.596252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:11.654948 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda90b1-4e01-416b-99c7-80c7654d9dfc admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:11.659994 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e6adf-4ac6-4eab-9b70-8395311a30d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:11.661262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e6adf-4ac6-4eab-9b70-8395311a30d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:11.661667 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e6adf-4ac6-4eab-9b70-8395311a30d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:11.662253 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e6adf-4ac6-4eab-9b70-8395311a30d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:11.681355 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e6adf-4ac6-4eab-9b70-8395311a30d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:11.683267 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e6adf-4ac6-4eab-9b70-8395311a30d4 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:11.695558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 998/1979] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:12.162273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8017961-6765-45aa-8062-f75f5de9a31f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:12.163838 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e8017961-6765-45aa-8062-f75f5de9a31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:12.176605 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8017961-6765-45aa-8062-f75f5de9a31f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:12.181786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f23a26-7080-48c3-b32c-8ceca57ebe7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:12.182818 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f23a26-7080-48c3-b32c-8ceca57ebe7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:12.183231 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f23a26-7080-48c3-b32c-8ceca57ebe7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:12.183568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f23a26-7080-48c3-b32c-8ceca57ebe7c admin admin] PATH_INFO: `/v3/projects/c4a9185149e24188b3bc35dadc176db4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:12.185137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f23a26-7080-48c3-b32c-8ceca57ebe7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4a9185149e24188b3bc35dadc176db4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:12.187439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f23a26-7080-48c3-b32c-8ceca57ebe7c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:12.275456 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-10f23a26-7080-48c3-b32c-8ceca57ebe7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4a9185149e24188b3bc35dadc176db4', 'request_id': 'req-10f23a26-7080-48c3-b32c-8ceca57ebe7c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:12.285189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-10f23a26-7080-48c3-b32c-8ceca57ebe7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4a9185149e24188b3bc35dadc176db4', 'request_id': 'req-10f23a26-7080-48c3-b32c-8ceca57ebe7c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:12.291907 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-10f23a26-7080-48c3-b32c-8ceca57ebe7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4a9185149e24188b3bc35dadc176db4', 'request_id': 'req-10f23a26-7080-48c3-b32c-8ceca57ebe7c'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:12.312559 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-10f23a26-7080-48c3-b32c-8ceca57ebe7c admin admin] The token cache is being invalidate because project c4a9185149e24188b3bc35dadc176db4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:37:12.312559 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-10f23a26-7080-48c3-b32c-8ceca57ebe7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:12.312559 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-10f23a26-7080-48c3-b32c-8ceca57ebe7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:12.312559 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 982/1980] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:37:12 2022] DELETE /identity/v3/projects/c4a9185149e24188b3bc35dadc176db4 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:37:12.312966 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03769b-90d7-433d-a8ea-707ee39ecd39 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:12.312966 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03769b-90d7-433d-a8ea-707ee39ecd39 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:12.312966 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03769b-90d7-433d-a8ea-707ee39ecd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:12.330488 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-2e03769b-90d7-433d-a8ea-707ee39ecd39 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:12.331729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2e03769b-90d7-433d-a8ea-707ee39ecd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:12.398087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 999/1981] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:12.406525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029a22f-3d12-4669-a4c9-16e7b19ff82f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:12.406824 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029a22f-3d12-4669-a4c9-16e7b19ff82f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:12.407157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029a22f-3d12-4669-a4c9-16e7b19ff82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:12.428141 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3029a22f-3d12-4669-a4c9-16e7b19ff82f None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:12.429362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3029a22f-3d12-4669-a4c9-16e7b19ff82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:12.439021 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 983/1982] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:12.443314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03769b-90d7-433d-a8ea-707ee39ecd39 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:12.453608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03769b-90d7-433d-a8ea-707ee39ecd39 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:12.458471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f923573-a1cb-45aa-a690-1d49cfb47dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:12.459179 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f923573-a1cb-45aa-a690-1d49cfb47dfb admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:12.459482 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f923573-a1cb-45aa-a690-1d49cfb47dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:12.459765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f923573-a1cb-45aa-a690-1d49cfb47dfb admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:12.460499 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f923573-a1cb-45aa-a690-1d49cfb47dfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:37:12.460850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f923573-a1cb-45aa-a690-1d49cfb47dfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:12.462514 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f923573-a1cb-45aa-a690-1d49cfb47dfb admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:12.476672 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1000/1983] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:37:12.481144 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029a22f-3d12-4669-a4c9-16e7b19ff82f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:12.492401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029a22f-3d12-4669-a4c9-16e7b19ff82f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:12.498697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693a2b4-8961-4897-9e4b-9bdd3c4cd028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:12.499530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693a2b4-8961-4897-9e4b-9bdd3c4cd028 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:12.499890 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693a2b4-8961-4897-9e4b-9bdd3c4cd028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:12.500300 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693a2b4-8961-4897-9e4b-9bdd3c4cd028 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:12.501025 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693a2b4-8961-4897-9e4b-9bdd3c4cd028 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:12.503242 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693a2b4-8961-4897-9e4b-9bdd3c4cd028 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:12.535319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 984/1984] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:37:12 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:12.536667 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f923573-a1cb-45aa-a690-1d49cfb47dfb admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:12.546813 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f923573-a1cb-45aa-a690-1d49cfb47dfb admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:12.604917 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d1eaa-6535-4c87-97d7-c3fdbdb42b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:12.605589 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d1eaa-6535-4c87-97d7-c3fdbdb42b1f admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:12.605875 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d1eaa-6535-4c87-97d7-c3fdbdb42b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:12.606223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d1eaa-6535-4c87-97d7-c3fdbdb42b1f admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:12.606912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d1eaa-6535-4c87-97d7-c3fdbdb42b1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:12.608841 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d1eaa-6535-4c87-97d7-c3fdbdb42b1f admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:12.630310 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1001/1985] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:37:12 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:12.634395 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693a2b4-8961-4897-9e4b-9bdd3c4cd028 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:12.645840 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693a2b4-8961-4897-9e4b-9bdd3c4cd028 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:12.650484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6ba03-af78-4be4-a356-01fc6b5a8040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:12.651224 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6ba03-af78-4be4-a356-01fc6b5a8040 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:12.651518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6ba03-af78-4be4-a356-01fc6b5a8040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:12.651807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6ba03-af78-4be4-a356-01fc6b5a8040 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:12.652542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6ba03-af78-4be4-a356-01fc6b5a8040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:12.654447 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6ba03-af78-4be4-a356-01fc6b5a8040 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:12.665335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 985/1986] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:12.669388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d1eaa-6535-4c87-97d7-c3fdbdb42b1f admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:12.679443 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d1eaa-6535-4c87-97d7-c3fdbdb42b1f admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:12.685228 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40646379-c327-4476-8654-23cd91b98f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:12.685917 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40646379-c327-4476-8654-23cd91b98f9a admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:12.686239 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40646379-c327-4476-8654-23cd91b98f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:12.686549 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40646379-c327-4476-8654-23cd91b98f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:12.687306 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40646379-c327-4476-8654-23cd91b98f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:12.688907 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40646379-c327-4476-8654-23cd91b98f9a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:12.702981 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1002/1987] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:12.704093 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6ba03-af78-4be4-a356-01fc6b5a8040 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:12.714531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6ba03-af78-4be4-a356-01fc6b5a8040 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:12.720046 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4123a-0d9a-408b-b5fd-700fe15aaf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:12.720971 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4123a-0d9a-408b-b5fd-700fe15aaf99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:12.721371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4123a-0d9a-408b-b5fd-700fe15aaf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:12.721661 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4123a-0d9a-408b-b5fd-700fe15aaf99 admin admin] PATH_INFO: `/v3/projects/3dd4c9c12f6845f0b8e0e030a2d87e9d/users/05fa935ebbca444b9cc7457b605a3e3c/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:12.737859 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4123a-0d9a-408b-b5fd-700fe15aaf99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dd4c9c12f6845f0b8e0e030a2d87e9d, user_id=05fa935ebbca444b9cc7457b605a3e3c, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:12.740605 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4123a-0d9a-408b-b5fd-700fe15aaf99 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:12.753942 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 986/1988] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:12 2022] PUT /identity/v3/projects/3dd4c9c12f6845f0b8e0e030a2d87e9d/users/05fa935ebbca444b9cc7457b605a3e3c/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:37:12.764271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535eae9-072e-4483-a182-82b3c423dac9 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:12.764590 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535eae9-072e-4483-a182-82b3c423dac9 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:12.764869 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4535eae9-072e-4483-a182-82b3c423dac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:12.799095 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-4535eae9-072e-4483-a182-82b3c423dac9 None None] MFA Rules not processed for user `05fa935ebbca444b9cc7457b605a3e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:12.800302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-4535eae9-072e-4483-a182-82b3c423dac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:12.828994 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:12.871097 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1003/1989] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:12.883071 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4123a-0d9a-408b-b5fd-700fe15aaf99 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:12.884877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-09f4123a-0d9a-408b-b5fd-700fe15aaf99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:12.911767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:12.944512 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4123a-0d9a-408b-b5fd-700fe15aaf99 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:12.948108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d821c-b67c-43b0-b460-094732ab4904 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36de21e0b3de48dabe0f27a9db8a37aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:12.948752 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d821c-b67c-43b0-b460-094732ab4904 service glance] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:12.949041 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d821c-b67c-43b0-b460-094732ab4904 service glance] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:12.949321 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d821c-b67c-43b0-b460-094732ab4904 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:12.957667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d821c-b67c-43b0-b460-094732ab4904 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:12.959105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d821c-b67c-43b0-b460-094732ab4904 service glance] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:12.967919 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 987/1990] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:12 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:22.868195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=16 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:22.963812 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=16 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:30.641739 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535eae9-072e-4483-a182-82b3c423dac9 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:30.643789 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-4535eae9-072e-4483-a182-82b3c423dac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:30.721152 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4535eae9-072e-4483-a182-82b3c423dac9 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:30.725701 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84731391-8fe1-4c26-8968-3babe0611abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:30.726574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84731391-8fe1-4c26-8968-3babe0611abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:30.726861 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84731391-8fe1-4c26-8968-3babe0611abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:30.727186 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84731391-8fe1-4c26-8968-3babe0611abc admin admin] PATH_INFO: `/v3/users/53c519b28b174ef385dd7e76b85cf504` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:30.728366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84731391-8fe1-4c26-8968-3babe0611abc admin admin] RBAC: Authorizing `identity:delete_user(user_id=53c519b28b174ef385dd7e76b85cf504)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:30.730297 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84731391-8fe1-4c26-8968-3babe0611abc admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:30.777534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-84731391-8fe1-4c26-8968-3babe0611abc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53c519b28b174ef385dd7e76b85cf504'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:30.789292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-84731391-8fe1-4c26-8968-3babe0611abc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53c519b28b174ef385dd7e76b85cf504'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:30.799147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-84731391-8fe1-4c26-8968-3babe0611abc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53c519b28b174ef385dd7e76b85cf504'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:30.806898 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-84731391-8fe1-4c26-8968-3babe0611abc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53c519b28b174ef385dd7e76b85cf504'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:30.807665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-84731391-8fe1-4c26-8968-3babe0611abc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53c519b28b174ef385dd7e76b85cf504'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:30.814380 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1004/1991] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:37:30 2022] DELETE /identity/v3/users/53c519b28b174ef385dd7e76b85cf504 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:31.004068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d821c-b67c-43b0-b460-094732ab4904 service glance] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:31.016807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d821c-b67c-43b0-b460-094732ab4904 service glance] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:31.076258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cce967-0849-443a-951e-50174727969a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:31.077146 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cce967-0849-443a-951e-50174727969a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:31.077436 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cce967-0849-443a-951e-50174727969a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:31.077719 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cce967-0849-443a-951e-50174727969a admin admin] PATH_INFO: `/v3/projects/becdae52e5c64b1ea2bb05833505b0f8` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:31.078912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cce967-0849-443a-951e-50174727969a admin admin] RBAC: Authorizing `identity:delete_project(project_id=becdae52e5c64b1ea2bb05833505b0f8)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:31.080840 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cce967-0849-443a-951e-50174727969a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:31.136419 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-14cce967-0849-443a-951e-50174727969a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'becdae52e5c64b1ea2bb05833505b0f8', 'request_id': 'req-14cce967-0849-443a-951e-50174727969a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:31.145044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-14cce967-0849-443a-951e-50174727969a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'becdae52e5c64b1ea2bb05833505b0f8', 'request_id': 'req-14cce967-0849-443a-951e-50174727969a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:31.150696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-14cce967-0849-443a-951e-50174727969a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'becdae52e5c64b1ea2bb05833505b0f8', 'request_id': 'req-14cce967-0849-443a-951e-50174727969a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:31.162915 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-14cce967-0849-443a-951e-50174727969a admin admin] The token cache is being invalidate because project becdae52e5c64b1ea2bb05833505b0f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:37:31.163235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-14cce967-0849-443a-951e-50174727969a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:31.163873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-14cce967-0849-443a-951e-50174727969a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:31.165439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 988/1992] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:37:31 2022] DELETE /identity/v3/projects/becdae52e5c64b1ea2bb05833505b0f8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:31.175922 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091f171-cb8f-4d4a-a983-3345aebff456 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:31.176217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091f171-cb8f-4d4a-a983-3345aebff456 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:31.176524 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091f171-cb8f-4d4a-a983-3345aebff456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:31.194424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-3091f171-cb8f-4d4a-a983-3345aebff456 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:31.195602 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3091f171-cb8f-4d4a-a983-3345aebff456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:31.257007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1005/1993] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:31.266674 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e744579-867e-4a11-988a-d50a95346e42 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:31.267015 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e744579-867e-4a11-988a-d50a95346e42 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:31.267336 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e744579-867e-4a11-988a-d50a95346e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:31.287126 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3e744579-867e-4a11-988a-d50a95346e42 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:31.288352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3e744579-867e-4a11-988a-d50a95346e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:31.297104 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 989/1994] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:31.302254 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091f171-cb8f-4d4a-a983-3345aebff456 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:31.314272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091f171-cb8f-4d4a-a983-3345aebff456 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:31.319076 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02958f1-1a61-4f3f-b12a-22cfc740ec43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:31.319762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02958f1-1a61-4f3f-b12a-22cfc740ec43 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:31.320109 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02958f1-1a61-4f3f-b12a-22cfc740ec43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:31.320398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02958f1-1a61-4f3f-b12a-22cfc740ec43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:31.321136 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02958f1-1a61-4f3f-b12a-22cfc740ec43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:37:31.321452 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02958f1-1a61-4f3f-b12a-22cfc740ec43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:31.323208 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02958f1-1a61-4f3f-b12a-22cfc740ec43 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:31.335490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1006/1995] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:37:31.339801 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e744579-867e-4a11-988a-d50a95346e42 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:31.350892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e744579-867e-4a11-988a-d50a95346e42 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:31.355773 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163ec29-9cbb-43b1-b63e-e68a195b2300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:31.357047 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163ec29-9cbb-43b1-b63e-e68a195b2300 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:31.357335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163ec29-9cbb-43b1-b63e-e68a195b2300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:31.357621 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163ec29-9cbb-43b1-b63e-e68a195b2300 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:31.358327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163ec29-9cbb-43b1-b63e-e68a195b2300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:31.360257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163ec29-9cbb-43b1-b63e-e68a195b2300 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:31.388760 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 990/1996] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:37:31 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:31.393027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02958f1-1a61-4f3f-b12a-22cfc740ec43 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:31.407682 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02958f1-1a61-4f3f-b12a-22cfc740ec43 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:31.418858 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:31.478614 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2de39f-5bba-4e9f-94ad-4c7b1f5edc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:31.479345 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2de39f-5bba-4e9f-94ad-4c7b1f5edc3d admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:31.479633 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2de39f-5bba-4e9f-94ad-4c7b1f5edc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:31.479916 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2de39f-5bba-4e9f-94ad-4c7b1f5edc3d admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:31.480610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2de39f-5bba-4e9f-94ad-4c7b1f5edc3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:31.483767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2de39f-5bba-4e9f-94ad-4c7b1f5edc3d admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:31.508126 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1007/1997] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:37:31 2022] POST /identity/v3/users => generated 282 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:31.511758 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163ec29-9cbb-43b1-b63e-e68a195b2300 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:31.522909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163ec29-9cbb-43b1-b63e-e68a195b2300 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:31.532488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e51cc2-78a1-4226-b7fb-aa2dd6471251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:31.533179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e51cc2-78a1-4226-b7fb-aa2dd6471251 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:31.533781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e51cc2-78a1-4226-b7fb-aa2dd6471251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:31.534381 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e51cc2-78a1-4226-b7fb-aa2dd6471251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:31.535489 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e51cc2-78a1-4226-b7fb-aa2dd6471251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:31.537117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e51cc2-78a1-4226-b7fb-aa2dd6471251 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:31.550018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 991/1998] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:37:31.553871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2de39f-5bba-4e9f-94ad-4c7b1f5edc3d admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:31.564907 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2de39f-5bba-4e9f-94ad-4c7b1f5edc3d admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:31.569909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3cb05d-1614-4046-aed4-e2e16cedba5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:31.571068 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cb05d-1614-4046-aed4-e2e16cedba5c admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:31.571532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cb05d-1614-4046-aed4-e2e16cedba5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:31.572098 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cb05d-1614-4046-aed4-e2e16cedba5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:31.573470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3cb05d-1614-4046-aed4-e2e16cedba5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:31.576719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3cb05d-1614-4046-aed4-e2e16cedba5c admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:31.588786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1008/1999] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:31.593211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e51cc2-78a1-4226-b7fb-aa2dd6471251 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:31.603385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e51cc2-78a1-4226-b7fb-aa2dd6471251 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:31.608416 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3daf9e0-3b42-4deb-a74f-d7d718d04331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:31.609319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daf9e0-3b42-4deb-a74f-d7d718d04331 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:31.609608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daf9e0-3b42-4deb-a74f-d7d718d04331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:31.609924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daf9e0-3b42-4deb-a74f-d7d718d04331 admin admin] PATH_INFO: `/v3/projects/371b40c92338429b8beea50b1c9c58eb/users/f02f9eb9ed704e288de0a2a0416f5fd5/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:31.626580 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3daf9e0-3b42-4deb-a74f-d7d718d04331 admin admin] RBAC: Authorizing `identity:create_grant(project_id=371b40c92338429b8beea50b1c9c58eb, user_id=f02f9eb9ed704e288de0a2a0416f5fd5, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:31.628915 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3daf9e0-3b42-4deb-a74f-d7d718d04331 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:31.645492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 992/2000] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:31 2022] PUT /identity/v3/projects/371b40c92338429b8beea50b1c9c58eb/users/f02f9eb9ed704e288de0a2a0416f5fd5/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:31.656521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8610e-4f5d-465c-9d25-6a659e477328 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:31.657044 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8610e-4f5d-465c-9d25-6a659e477328 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:31.657176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8610e-4f5d-465c-9d25-6a659e477328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:31.698479 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-16e8610e-4f5d-465c-9d25-6a659e477328 None None] MFA Rules not processed for user `f02f9eb9ed704e288de0a2a0416f5fd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:31.700312 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-16e8610e-4f5d-465c-9d25-6a659e477328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:31.780611 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1009/2001] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:31 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:31.785200 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3daf9e0-3b42-4deb-a74f-d7d718d04331 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:31.820279 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:31.859610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3daf9e0-3b42-4deb-a74f-d7d718d04331 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:31.864231 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2667fb-2710-48a4-8349-347e4fa3fac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:31.864898 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2667fb-2710-48a4-8349-347e4fa3fac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:31.865184 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2667fb-2710-48a4-8349-347e4fa3fac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:31.865464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2667fb-2710-48a4-8349-347e4fa3fac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:31.866166 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2667fb-2710-48a4-8349-347e4fa3fac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:31.868127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2667fb-2710-48a4-8349-347e4fa3fac1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:31.894831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 993/2002] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:37:31 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:31.898886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8610e-4f5d-465c-9d25-6a659e477328 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:31.910531 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8610e-4f5d-465c-9d25-6a659e477328 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:31.975882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa2676-9195-4206-9d41-bb2a42d34bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:31.976574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa2676-9195-4206-9d41-bb2a42d34bdd admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:31.976858 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa2676-9195-4206-9d41-bb2a42d34bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:31.977136 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa2676-9195-4206-9d41-bb2a42d34bdd admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:31.977824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa2676-9195-4206-9d41-bb2a42d34bdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:31.979777 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa2676-9195-4206-9d41-bb2a42d34bdd admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:32.000744 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1010/2003] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:37:31 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:32.004565 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2667fb-2710-48a4-8349-347e4fa3fac1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:32.015825 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2667fb-2710-48a4-8349-347e4fa3fac1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:32.020885 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38dd88a-1e4e-4fa3-8e9a-274a26b2f51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:32.021569 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38dd88a-1e4e-4fa3-8e9a-274a26b2f51e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:32.021857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38dd88a-1e4e-4fa3-8e9a-274a26b2f51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:32.022142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38dd88a-1e4e-4fa3-8e9a-274a26b2f51e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:32.022878 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38dd88a-1e4e-4fa3-8e9a-274a26b2f51e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:32.024514 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38dd88a-1e4e-4fa3-8e9a-274a26b2f51e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:32.035882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 994/2004] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:32.040616 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa2676-9195-4206-9d41-bb2a42d34bdd admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:32.050674 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:32.055080 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa2676-9195-4206-9d41-bb2a42d34bdd admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:32.062082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79686c0-06d0-4f5f-83ee-a847d9f9bf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:32.063028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79686c0-06d0-4f5f-83ee-a847d9f9bf54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:32.063320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79686c0-06d0-4f5f-83ee-a847d9f9bf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:32.063599 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79686c0-06d0-4f5f-83ee-a847d9f9bf54 admin admin] PATH_INFO: `/v3/projects/bfd5d4c85ba742ecb6ddbf1c5f79e9bb/users/80f43913b48241f18863045d3996066d/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:32.078800 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79686c0-06d0-4f5f-83ee-a847d9f9bf54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfd5d4c85ba742ecb6ddbf1c5f79e9bb, user_id=80f43913b48241f18863045d3996066d, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:32.081470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79686c0-06d0-4f5f-83ee-a847d9f9bf54 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:32.094668 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1011/2005] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:32 2022] PUT /identity/v3/projects/bfd5d4c85ba742ecb6ddbf1c5f79e9bb/users/80f43913b48241f18863045d3996066d/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:32.098632 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38dd88a-1e4e-4fa3-8e9a-274a26b2f51e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:32.109950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38dd88a-1e4e-4fa3-8e9a-274a26b2f51e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:32.171564 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb53984-25d6-435f-991d-3128fea63c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:32.172233 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb53984-25d6-435f-991d-3128fea63c4f admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:32.172521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb53984-25d6-435f-991d-3128fea63c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:32.172802 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb53984-25d6-435f-991d-3128fea63c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:32.173525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb53984-25d6-435f-991d-3128fea63c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:32.175154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb53984-25d6-435f-991d-3128fea63c4f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:32.188394 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 995/2006] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:32.191567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79686c0-06d0-4f5f-83ee-a847d9f9bf54 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:32.202514 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79686c0-06d0-4f5f-83ee-a847d9f9bf54 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:32.208387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3e15d-f2f4-4d62-8fa3-d30107b5c51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:32.209396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3e15d-f2f4-4d62-8fa3-d30107b5c51c admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:32.209716 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3e15d-f2f4-4d62-8fa3-d30107b5c51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:32.210028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3e15d-f2f4-4d62-8fa3-d30107b5c51c admin admin] PATH_INFO: `/v3/projects/bfd5d4c85ba742ecb6ddbf1c5f79e9bb/users/80f43913b48241f18863045d3996066d/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:32.212232 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3e15d-f2f4-4d62-8fa3-d30107b5c51c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfd5d4c85ba742ecb6ddbf1c5f79e9bb, user_id=80f43913b48241f18863045d3996066d, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:32.214968 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3e15d-f2f4-4d62-8fa3-d30107b5c51c admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:32.226495 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1012/2007] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:32 2022] PUT /identity/v3/projects/bfd5d4c85ba742ecb6ddbf1c5f79e9bb/users/80f43913b48241f18863045d3996066d/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:32.237935 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71b659-43f0-42cc-89be-46bee7abcbf1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:32.238231 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71b659-43f0-42cc-89be-46bee7abcbf1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:32.238516 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71b659-43f0-42cc-89be-46bee7abcbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:32.265808 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0d71b659-43f0-42cc-89be-46bee7abcbf1 None None] MFA Rules not processed for user `80f43913b48241f18863045d3996066d`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:32.266984 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0d71b659-43f0-42cc-89be-46bee7abcbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:32.345105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 996/2008] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:32 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:32.356491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3e15d-f2f4-4d62-8fa3-d30107b5c51c admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:32.358181 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-90e3e15d-f2f4-4d62-8fa3-d30107b5c51c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:32.419503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3e15d-f2f4-4d62-8fa3-d30107b5c51c admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:32.423319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59934cba-30b4-443e-be2f-426a1de6e934 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:32.423969 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934cba-30b4-443e-be2f-426a1de6e934 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:32.424266 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934cba-30b4-443e-be2f-426a1de6e934 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:32.424541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934cba-30b4-443e-be2f-426a1de6e934 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:32.434731 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59934cba-30b4-443e-be2f-426a1de6e934 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:32.436266 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59934cba-30b4-443e-be2f-426a1de6e934 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:32.446406 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1013/2009] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:32 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:32.932219 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71b659-43f0-42cc-89be-46bee7abcbf1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:32.959558 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:33.001947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71b659-43f0-42cc-89be-46bee7abcbf1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:33.006475 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd309b2-8195-4ada-be74-3367bbf38568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:33.007391 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd309b2-8195-4ada-be74-3367bbf38568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:33.007677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd309b2-8195-4ada-be74-3367bbf38568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:33.007957 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd309b2-8195-4ada-be74-3367bbf38568 admin admin] PATH_INFO: `/v3/users/f02f9eb9ed704e288de0a2a0416f5fd5` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:33.009149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd309b2-8195-4ada-be74-3367bbf38568 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f02f9eb9ed704e288de0a2a0416f5fd5)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:33.011132 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd309b2-8195-4ada-be74-3367bbf38568 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:33.064072 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9fd309b2-8195-4ada-be74-3367bbf38568 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f02f9eb9ed704e288de0a2a0416f5fd5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:33.071662 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9fd309b2-8195-4ada-be74-3367bbf38568 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f02f9eb9ed704e288de0a2a0416f5fd5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:33.079801 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9fd309b2-8195-4ada-be74-3367bbf38568 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f02f9eb9ed704e288de0a2a0416f5fd5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:33.080441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9fd309b2-8195-4ada-be74-3367bbf38568 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f02f9eb9ed704e288de0a2a0416f5fd5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:33.081054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9fd309b2-8195-4ada-be74-3367bbf38568 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f02f9eb9ed704e288de0a2a0416f5fd5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:33.101932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 997/2010] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:37:32 2022] DELETE /identity/v3/users/f02f9eb9ed704e288de0a2a0416f5fd5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:33.107681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633de59-87b7-4255-bf78-53195a3de8eb None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:33.108259 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633de59-87b7-4255-bf78-53195a3de8eb None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:33.108353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633de59-87b7-4255-bf78-53195a3de8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:33.129054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-2633de59-87b7-4255-bf78-53195a3de8eb None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:33.130346 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2633de59-87b7-4255-bf78-53195a3de8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:33.226511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1014/2011] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:33.236278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd309b2-8195-4ada-be74-3367bbf38568 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:33.237702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9fd309b2-8195-4ada-be74-3367bbf38568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:33.307943 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd309b2-8195-4ada-be74-3367bbf38568 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:33.312389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4e1c7-b517-45f9-ab9b-f675124b3519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:33.313057 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4e1c7-b517-45f9-ab9b-f675124b3519 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:33.313349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4e1c7-b517-45f9-ab9b-f675124b3519 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:33.313637 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4e1c7-b517-45f9-ab9b-f675124b3519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:33.323754 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4e1c7-b517-45f9-ab9b-f675124b3519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:33.326422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4e1c7-b517-45f9-ab9b-f675124b3519 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:33.337740 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 998/2012] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:33.642011 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633de59-87b7-4255-bf78-53195a3de8eb None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:33.643490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2633de59-87b7-4255-bf78-53195a3de8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:33.666075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633de59-87b7-4255-bf78-53195a3de8eb None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:33.672126 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d7bc0-381a-4077-955e-70beeef9258c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:33.673008 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d7bc0-381a-4077-955e-70beeef9258c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:33.673292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d7bc0-381a-4077-955e-70beeef9258c admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:33.673611 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d7bc0-381a-4077-955e-70beeef9258c admin admin] PATH_INFO: `/v3/projects/371b40c92338429b8beea50b1c9c58eb` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:33.674773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d7bc0-381a-4077-955e-70beeef9258c admin admin] RBAC: Authorizing `identity:delete_project(project_id=371b40c92338429b8beea50b1c9c58eb)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:33.676778 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d7bc0-381a-4077-955e-70beeef9258c admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:33.731742 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7b3d7bc0-381a-4077-955e-70beeef9258c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '371b40c92338429b8beea50b1c9c58eb', 'request_id': 'req-7b3d7bc0-381a-4077-955e-70beeef9258c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:33.737847 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7b3d7bc0-381a-4077-955e-70beeef9258c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '371b40c92338429b8beea50b1c9c58eb', 'request_id': 'req-7b3d7bc0-381a-4077-955e-70beeef9258c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:33.744542 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7b3d7bc0-381a-4077-955e-70beeef9258c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '371b40c92338429b8beea50b1c9c58eb', 'request_id': 'req-7b3d7bc0-381a-4077-955e-70beeef9258c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:33.748736 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:33.757701 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7b3d7bc0-381a-4077-955e-70beeef9258c admin admin] The token cache is being invalidate because project 371b40c92338429b8beea50b1c9c58eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:37:33.758009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7b3d7bc0-381a-4077-955e-70beeef9258c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:33.758968 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7b3d7bc0-381a-4077-955e-70beeef9258c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:33.760916 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1015/2013] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:37:33 2022] DELETE /identity/v3/projects/371b40c92338429b8beea50b1c9c58eb => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:33.764960 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4e1c7-b517-45f9-ab9b-f675124b3519 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:33.774640 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4e1c7-b517-45f9-ab9b-f675124b3519 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:33.828644 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:33.853835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442e3b2-c8ef-4bec-a614-592386f1949b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:33.854863 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442e3b2-c8ef-4bec-a614-592386f1949b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:33.855193 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442e3b2-c8ef-4bec-a614-592386f1949b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:33.855517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442e3b2-c8ef-4bec-a614-592386f1949b admin admin] PATH_INFO: `/v3/users/80f43913b48241f18863045d3996066d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:33.857615 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442e3b2-c8ef-4bec-a614-592386f1949b admin admin] RBAC: Authorizing `identity:delete_user(user_id=80f43913b48241f18863045d3996066d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:33.860044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442e3b2-c8ef-4bec-a614-592386f1949b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:33.928450 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8442e3b2-c8ef-4bec-a614-592386f1949b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80f43913b48241f18863045d3996066d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:33.937053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8442e3b2-c8ef-4bec-a614-592386f1949b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80f43913b48241f18863045d3996066d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:33.946227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8442e3b2-c8ef-4bec-a614-592386f1949b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80f43913b48241f18863045d3996066d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:33.946945 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8442e3b2-c8ef-4bec-a614-592386f1949b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80f43913b48241f18863045d3996066d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:33.947565 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8442e3b2-c8ef-4bec-a614-592386f1949b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80f43913b48241f18863045d3996066d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:33.978616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 999/2014] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:37:33 2022] DELETE /identity/v3/users/80f43913b48241f18863045d3996066d => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:34.364011 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d7bc0-381a-4077-955e-70beeef9258c admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:34.366006 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7b3d7bc0-381a-4077-955e-70beeef9258c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:34.381984 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d7bc0-381a-4077-955e-70beeef9258c admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:34.443340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1af3b-5c7b-4348-8732-9a97ac0c37c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:34.444211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1af3b-5c7b-4348-8732-9a97ac0c37c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:34.444508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1af3b-5c7b-4348-8732-9a97ac0c37c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:34.444775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1af3b-5c7b-4348-8732-9a97ac0c37c4 admin admin] PATH_INFO: `/v3/projects/bfd5d4c85ba742ecb6ddbf1c5f79e9bb` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:34.445961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1af3b-5c7b-4348-8732-9a97ac0c37c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfd5d4c85ba742ecb6ddbf1c5f79e9bb)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:34.447912 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1af3b-5c7b-4348-8732-9a97ac0c37c4 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:34.492637 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-eaa1af3b-5c7b-4348-8732-9a97ac0c37c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfd5d4c85ba742ecb6ddbf1c5f79e9bb', 'request_id': 'req-eaa1af3b-5c7b-4348-8732-9a97ac0c37c4'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:34.499764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-eaa1af3b-5c7b-4348-8732-9a97ac0c37c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfd5d4c85ba742ecb6ddbf1c5f79e9bb', 'request_id': 'req-eaa1af3b-5c7b-4348-8732-9a97ac0c37c4'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:34.507763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-eaa1af3b-5c7b-4348-8732-9a97ac0c37c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfd5d4c85ba742ecb6ddbf1c5f79e9bb', 'request_id': 'req-eaa1af3b-5c7b-4348-8732-9a97ac0c37c4'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:34.519806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-eaa1af3b-5c7b-4348-8732-9a97ac0c37c4 admin admin] The token cache is being invalidate because project bfd5d4c85ba742ecb6ddbf1c5f79e9bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:37:34.520397 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-eaa1af3b-5c7b-4348-8732-9a97ac0c37c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:34.520953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-eaa1af3b-5c7b-4348-8732-9a97ac0c37c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:34.523170 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1016/2015] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:37:34 2022] DELETE /identity/v3/projects/bfd5d4c85ba742ecb6ddbf1c5f79e9bb => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:34.528768 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d44085-0365-4862-a878-c9ed182ac9f3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:34.529260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d44085-0365-4862-a878-c9ed182ac9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:34.529369 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d44085-0365-4862-a878-c9ed182ac9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:34.551339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-f3d44085-0365-4862-a878-c9ed182ac9f3 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:34.552712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f3d44085-0365-4862-a878-c9ed182ac9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:34.618075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1000/2016] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:34.627030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19118f-c3e1-4468-84c4-efa3967ffdce None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:34.627331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19118f-c3e1-4468-84c4-efa3967ffdce None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:34.627622 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19118f-c3e1-4468-84c4-efa3967ffdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:34.647158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-fb19118f-c3e1-4468-84c4-efa3967ffdce None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:34.648458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-fb19118f-c3e1-4468-84c4-efa3967ffdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:34.658482 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1017/2017] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:34.664059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d44085-0365-4862-a878-c9ed182ac9f3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:34.676966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d44085-0365-4862-a878-c9ed182ac9f3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:34.683390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881199bc-831d-44d6-b9b9-4d5f4ce46005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:34.685345 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881199bc-831d-44d6-b9b9-4d5f4ce46005 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:34.685831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881199bc-831d-44d6-b9b9-4d5f4ce46005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:34.686301 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881199bc-831d-44d6-b9b9-4d5f4ce46005 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:34.687468 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881199bc-831d-44d6-b9b9-4d5f4ce46005 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:37:34.687979 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881199bc-831d-44d6-b9b9-4d5f4ce46005 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:34.689695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881199bc-831d-44d6-b9b9-4d5f4ce46005 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:34.703304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1001/2018] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:37:34.707050 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19118f-c3e1-4468-84c4-efa3967ffdce None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:34.718964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19118f-c3e1-4468-84c4-efa3967ffdce None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:34.725013 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a14a1d-2cd7-48a9-9f72-b1b6c7404dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:34.726176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a14a1d-2cd7-48a9-9f72-b1b6c7404dff admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:34.726869 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a14a1d-2cd7-48a9-9f72-b1b6c7404dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:34.727454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a14a1d-2cd7-48a9-9f72-b1b6c7404dff admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:34.728676 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a14a1d-2cd7-48a9-9f72-b1b6c7404dff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:34.731583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a14a1d-2cd7-48a9-9f72-b1b6c7404dff admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:34.758216 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1018/2019] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:37:34 2022] POST /identity/v3/projects => generated 355 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:37:34.762781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881199bc-831d-44d6-b9b9-4d5f4ce46005 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:34.772628 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881199bc-831d-44d6-b9b9-4d5f4ce46005 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:34.820912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:34.832527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a493cf4-4198-4c4c-99e8-c3c0cf770aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:34.833201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a493cf4-4198-4c4c-99e8-c3c0cf770aa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:34.833490 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a493cf4-4198-4c4c-99e8-c3c0cf770aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:34.833772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a493cf4-4198-4c4c-99e8-c3c0cf770aa7 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:34.834470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a493cf4-4198-4c4c-99e8-c3c0cf770aa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:34.836407 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a493cf4-4198-4c4c-99e8-c3c0cf770aa7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:34.859560 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1002/2020] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:37:34 2022] POST /identity/v3/users => generated 278 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:34.863254 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a14a1d-2cd7-48a9-9f72-b1b6c7404dff admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:34.874645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a14a1d-2cd7-48a9-9f72-b1b6c7404dff admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:34.879239 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279fa4eb-ea29-4cf0-aeb1-8f2696a9a1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:34.879911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fa4eb-ea29-4cf0-aeb1-8f2696a9a1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:34.880204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fa4eb-ea29-4cf0-aeb1-8f2696a9a1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:34.880491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279fa4eb-ea29-4cf0-aeb1-8f2696a9a1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:34.881229 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279fa4eb-ea29-4cf0-aeb1-8f2696a9a1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:34.882866 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279fa4eb-ea29-4cf0-aeb1-8f2696a9a1e8 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:34.893339 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1019/2021] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:34.897296 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a493cf4-4198-4c4c-99e8-c3c0cf770aa7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:34.907875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a493cf4-4198-4c4c-99e8-c3c0cf770aa7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:34.912819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b8b12-6330-431f-bbc8-1d9282fd797c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:34.913538 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b8b12-6330-431f-bbc8-1d9282fd797c admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:34.913881 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b8b12-6330-431f-bbc8-1d9282fd797c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:34.914171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b8b12-6330-431f-bbc8-1d9282fd797c admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:34.914903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b8b12-6330-431f-bbc8-1d9282fd797c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:34.916516 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b8b12-6330-431f-bbc8-1d9282fd797c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:34.927555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1003/2022] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:34.931518 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279fa4eb-ea29-4cf0-aeb1-8f2696a9a1e8 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:34.942210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279fa4eb-ea29-4cf0-aeb1-8f2696a9a1e8 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:34.946813 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c72d87-2d30-4c14-8726-209b099c7085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:34.947730 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c72d87-2d30-4c14-8726-209b099c7085 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:34.948019 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c72d87-2d30-4c14-8726-209b099c7085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:34.948301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c72d87-2d30-4c14-8726-209b099c7085 admin admin] PATH_INFO: `/v3/projects/5b37e6edb6044254afd6341775b46e17/users/7d8659a964d645ea99edec7bece0b7c1/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:34.962969 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c72d87-2d30-4c14-8726-209b099c7085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b37e6edb6044254afd6341775b46e17, user_id=7d8659a964d645ea99edec7bece0b7c1, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:34.966714 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c72d87-2d30-4c14-8726-209b099c7085 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:34.983317 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1020/2023] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:34 2022] PUT /identity/v3/projects/5b37e6edb6044254afd6341775b46e17/users/7d8659a964d645ea99edec7bece0b7c1/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:34.992379 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b87e42-e014-4696-b58b-755bea4006fb None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:34.992965 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b87e42-e014-4696-b58b-755bea4006fb None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:34.993292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b87e42-e014-4696-b58b-755bea4006fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:35.019976 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b3b87e42-e014-4696-b58b-755bea4006fb None None] MFA Rules not processed for user `7d8659a964d645ea99edec7bece0b7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:35.021184 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b3b87e42-e014-4696-b58b-755bea4006fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:35.088840 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1004/2024] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:34 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:35.093828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c72d87-2d30-4c14-8726-209b099c7085 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:35.118102 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:35.165455 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c72d87-2d30-4c14-8726-209b099c7085 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:35.170801 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddf932-8865-4d51-98d6-1d97884b3466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:35.171501 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddf932-8865-4d51-98d6-1d97884b3466 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:35.171794 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddf932-8865-4d51-98d6-1d97884b3466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:35.172079 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddf932-8865-4d51-98d6-1d97884b3466 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:35.172841 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ddf932-8865-4d51-98d6-1d97884b3466 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:35.174761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ddf932-8865-4d51-98d6-1d97884b3466 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:35.202855 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1021/2025] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:37:35 2022] POST /identity/v3/projects => generated 355 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:35.204895 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b87e42-e014-4696-b58b-755bea4006fb None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:35.216472 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b87e42-e014-4696-b58b-755bea4006fb None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:35.289187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880501e8-9866-4118-b7ee-8d4799be6e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:35.289894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880501e8-9866-4118-b7ee-8d4799be6e22 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:35.290186 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880501e8-9866-4118-b7ee-8d4799be6e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:35.290489 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880501e8-9866-4118-b7ee-8d4799be6e22 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:35.291250 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880501e8-9866-4118-b7ee-8d4799be6e22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:35.295285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880501e8-9866-4118-b7ee-8d4799be6e22 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:35.316821 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1005/2026] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:37:35 2022] POST /identity/v3/users => generated 284 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:35.321028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddf932-8865-4d51-98d6-1d97884b3466 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:35.334251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddf932-8865-4d51-98d6-1d97884b3466 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:35.355840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75dce1-14d7-402c-8cb6-1aca372d8b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:35.356692 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75dce1-14d7-402c-8cb6-1aca372d8b81 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:35.356957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75dce1-14d7-402c-8cb6-1aca372d8b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:35.357208 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75dce1-14d7-402c-8cb6-1aca372d8b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:35.357965 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75dce1-14d7-402c-8cb6-1aca372d8b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:35.359879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75dce1-14d7-402c-8cb6-1aca372d8b81 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:35.373423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1022/2027] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:35.377463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880501e8-9866-4118-b7ee-8d4799be6e22 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:35.388761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880501e8-9866-4118-b7ee-8d4799be6e22 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:35.394169 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facac1a9-0ce1-4699-b37b-a80423f54a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:35.395104 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facac1a9-0ce1-4699-b37b-a80423f54a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:35.395397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facac1a9-0ce1-4699-b37b-a80423f54a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:35.395834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facac1a9-0ce1-4699-b37b-a80423f54a8d admin admin] PATH_INFO: `/v3/projects/977203a35a3d4fb0b94566759835e600/users/236da07c2b7043419aed2d9e83493502/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:35.426708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facac1a9-0ce1-4699-b37b-a80423f54a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=977203a35a3d4fb0b94566759835e600, user_id=236da07c2b7043419aed2d9e83493502, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:35.429399 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facac1a9-0ce1-4699-b37b-a80423f54a8d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:35.445292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1006/2028] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:35 2022] PUT /identity/v3/projects/977203a35a3d4fb0b94566759835e600/users/236da07c2b7043419aed2d9e83493502/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:37:35.449314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75dce1-14d7-402c-8cb6-1aca372d8b81 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:35.462304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75dce1-14d7-402c-8cb6-1aca372d8b81 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:35.556186 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1be0b5-a5a0-4a76-bbff-173a47c4aeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:35.557068 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1be0b5-a5a0-4a76-bbff-173a47c4aeb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:35.557374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1be0b5-a5a0-4a76-bbff-173a47c4aeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:35.557718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1be0b5-a5a0-4a76-bbff-173a47c4aeb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:35.558512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1be0b5-a5a0-4a76-bbff-173a47c4aeb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:35.560306 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1be0b5-a5a0-4a76-bbff-173a47c4aeb0 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:35.573189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1023/2029] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:35.581366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facac1a9-0ce1-4699-b37b-a80423f54a8d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:35.593079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facac1a9-0ce1-4699-b37b-a80423f54a8d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:35.597823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec792327-4e7d-43ca-bb50-870c8ed3b998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:35.598710 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec792327-4e7d-43ca-bb50-870c8ed3b998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:35.599033 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec792327-4e7d-43ca-bb50-870c8ed3b998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:35.599318 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec792327-4e7d-43ca-bb50-870c8ed3b998 admin admin] PATH_INFO: `/v3/projects/977203a35a3d4fb0b94566759835e600/users/236da07c2b7043419aed2d9e83493502/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:35.601322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec792327-4e7d-43ca-bb50-870c8ed3b998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=977203a35a3d4fb0b94566759835e600, user_id=236da07c2b7043419aed2d9e83493502, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:35.603979 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec792327-4e7d-43ca-bb50-870c8ed3b998 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:35.614453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1007/2030] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:35 2022] PUT /identity/v3/projects/977203a35a3d4fb0b94566759835e600/users/236da07c2b7043419aed2d9e83493502/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:35.625029 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44d7eb-75e5-4bdf-8514-2c683d569401 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:35.625430 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44d7eb-75e5-4bdf-8514-2c683d569401 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:35.625718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44d7eb-75e5-4bdf-8514-2c683d569401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:35.657003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-4e44d7eb-75e5-4bdf-8514-2c683d569401 None None] MFA Rules not processed for user `236da07c2b7043419aed2d9e83493502`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:35.658220 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-4e44d7eb-75e5-4bdf-8514-2c683d569401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:35.737586 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1024/2031] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:35 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:35.747431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec792327-4e7d-43ca-bb50-870c8ed3b998 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:35.749599 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ec792327-4e7d-43ca-bb50-870c8ed3b998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:35.764370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:35.810892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec792327-4e7d-43ca-bb50-870c8ed3b998 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:35.814720 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabeb404-2276-44b0-a52a-eb8ab1a53445 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:35.815427 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabeb404-2276-44b0-a52a-eb8ab1a53445 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:35.815720 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabeb404-2276-44b0-a52a-eb8ab1a53445 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:35.816004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabeb404-2276-44b0-a52a-eb8ab1a53445 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:35.828347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabeb404-2276-44b0-a52a-eb8ab1a53445 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:35.830088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabeb404-2276-44b0-a52a-eb8ab1a53445 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:35.841715 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1008/2032] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:35 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:35.917214 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44d7eb-75e5-4bdf-8514-2c683d569401 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:35.927145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44d7eb-75e5-4bdf-8514-2c683d569401 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:35.931249 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c248cf3-dcba-45ac-885c-2d0ac23e85aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:35.931917 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c248cf3-dcba-45ac-885c-2d0ac23e85aa service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:35.932195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c248cf3-dcba-45ac-885c-2d0ac23e85aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:35.932497 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c248cf3-dcba-45ac-885c-2d0ac23e85aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:35.945665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:35.994901 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c248cf3-dcba-45ac-885c-2d0ac23e85aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:35.996981 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c248cf3-dcba-45ac-885c-2d0ac23e85aa service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:36.006442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1025/2033] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:35 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:37:37.066622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabeb404-2276-44b0-a52a-eb8ab1a53445 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:37.069139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cabeb404-2276-44b0-a52a-eb8ab1a53445 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:37.135661 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabeb404-2276-44b0-a52a-eb8ab1a53445 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:37.141208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b3eaa-cfce-441f-9152-403cb141a82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:37.142110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b3eaa-cfce-441f-9152-403cb141a82e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:37.142405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b3eaa-cfce-441f-9152-403cb141a82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:37.142695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b3eaa-cfce-441f-9152-403cb141a82e admin admin] PATH_INFO: `/v3/users/43596e8c32d041a0926976c255867a05` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:37.143948 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b3eaa-cfce-441f-9152-403cb141a82e admin admin] RBAC: Authorizing `identity:delete_user(user_id=43596e8c32d041a0926976c255867a05)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:37.145927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b3eaa-cfce-441f-9152-403cb141a82e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:37.194382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-601b3eaa-cfce-441f-9152-403cb141a82e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43596e8c32d041a0926976c255867a05'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:37.201829 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-601b3eaa-cfce-441f-9152-403cb141a82e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43596e8c32d041a0926976c255867a05'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:37.210803 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-601b3eaa-cfce-441f-9152-403cb141a82e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43596e8c32d041a0926976c255867a05'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:37.211487 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-601b3eaa-cfce-441f-9152-403cb141a82e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43596e8c32d041a0926976c255867a05'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:37.212102 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-601b3eaa-cfce-441f-9152-403cb141a82e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43596e8c32d041a0926976c255867a05'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:37.232155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1009/2034] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:37:37 2022] DELETE /identity/v3/users/43596e8c32d041a0926976c255867a05 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:37.236963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f481a25-7ee7-4f7d-bb66-5aae915e2070 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:37.237470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f481a25-7ee7-4f7d-bb66-5aae915e2070 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:37.237566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f481a25-7ee7-4f7d-bb66-5aae915e2070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:37.255989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-3f481a25-7ee7-4f7d-bb66-5aae915e2070 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:37.257193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3f481a25-7ee7-4f7d-bb66-5aae915e2070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:37.330662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1026/2035] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:37.340374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b3eaa-cfce-441f-9152-403cb141a82e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:37.341707 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-601b3eaa-cfce-441f-9152-403cb141a82e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:37.378170 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:37.405836 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b3eaa-cfce-441f-9152-403cb141a82e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:37.409788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0efa8-fe25-428c-b560-9f3b08cd1848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:37.410484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0efa8-fe25-428c-b560-9f3b08cd1848 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:37.410767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0efa8-fe25-428c-b560-9f3b08cd1848 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:37.411083 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0efa8-fe25-428c-b560-9f3b08cd1848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:37.420557 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0efa8-fe25-428c-b560-9f3b08cd1848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:37.421994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0efa8-fe25-428c-b560-9f3b08cd1848 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:37.432401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1010/2036] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:37.772234 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f481a25-7ee7-4f7d-bb66-5aae915e2070 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:37.773782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3f481a25-7ee7-4f7d-bb66-5aae915e2070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:37.795974 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f481a25-7ee7-4f7d-bb66-5aae915e2070 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:37.801836 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c9ebd-e35e-4e1f-b934-2a581d2ee285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:37.802567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c9ebd-e35e-4e1f-b934-2a581d2ee285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:37.802843 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c9ebd-e35e-4e1f-b934-2a581d2ee285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:37.803743 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c9ebd-e35e-4e1f-b934-2a581d2ee285 admin admin] PATH_INFO: `/v3/projects/c963ec19d64b44758a96a22c7c8c3450` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:37.804977 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c9ebd-e35e-4e1f-b934-2a581d2ee285 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c963ec19d64b44758a96a22c7c8c3450)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:37.806903 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c9ebd-e35e-4e1f-b934-2a581d2ee285 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:37.859931 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-1e0c9ebd-e35e-4e1f-b934-2a581d2ee285 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c963ec19d64b44758a96a22c7c8c3450', 'request_id': 'req-1e0c9ebd-e35e-4e1f-b934-2a581d2ee285'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:37.865964 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-1e0c9ebd-e35e-4e1f-b934-2a581d2ee285 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c963ec19d64b44758a96a22c7c8c3450', 'request_id': 'req-1e0c9ebd-e35e-4e1f-b934-2a581d2ee285'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:37.873953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-1e0c9ebd-e35e-4e1f-b934-2a581d2ee285 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c963ec19d64b44758a96a22c7c8c3450', 'request_id': 'req-1e0c9ebd-e35e-4e1f-b934-2a581d2ee285'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:37.890112 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-1e0c9ebd-e35e-4e1f-b934-2a581d2ee285 admin admin] The token cache is being invalidate because project c963ec19d64b44758a96a22c7c8c3450 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:37:37.890410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-1e0c9ebd-e35e-4e1f-b934-2a581d2ee285 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:37.891106 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-1e0c9ebd-e35e-4e1f-b934-2a581d2ee285 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:37.892838 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1027/2037] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:37:37 2022] DELETE /identity/v3/projects/c963ec19d64b44758a96a22c7c8c3450 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:37.896587 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0efa8-fe25-428c-b560-9f3b08cd1848 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:37.906651 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0efa8-fe25-428c-b560-9f3b08cd1848 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:37.972961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ac30d-07af-4940-bf56-5cd184508d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:37.973886 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ac30d-07af-4940-bf56-5cd184508d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:37.974196 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ac30d-07af-4940-bf56-5cd184508d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:37.974475 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ac30d-07af-4940-bf56-5cd184508d86 admin admin] PATH_INFO: `/v3/users/ad37ce5c18194991b8b908ecf276a135` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:37.975695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ac30d-07af-4940-bf56-5cd184508d86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad37ce5c18194991b8b908ecf276a135)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:37.977774 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ac30d-07af-4940-bf56-5cd184508d86 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:38.026232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-229ac30d-07af-4940-bf56-5cd184508d86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad37ce5c18194991b8b908ecf276a135'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:38.033565 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-229ac30d-07af-4940-bf56-5cd184508d86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad37ce5c18194991b8b908ecf276a135'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:38.042622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-229ac30d-07af-4940-bf56-5cd184508d86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad37ce5c18194991b8b908ecf276a135'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:38.043370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-229ac30d-07af-4940-bf56-5cd184508d86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad37ce5c18194991b8b908ecf276a135'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:38.044645 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-229ac30d-07af-4940-bf56-5cd184508d86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad37ce5c18194991b8b908ecf276a135'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:38.062462 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1011/2038] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:37:37 2022] DELETE /identity/v3/users/ad37ce5c18194991b8b908ecf276a135 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:37:38.408742 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c9ebd-e35e-4e1f-b934-2a581d2ee285 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:38.410582 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1e0c9ebd-e35e-4e1f-b934-2a581d2ee285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:38.430469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c9ebd-e35e-4e1f-b934-2a581d2ee285 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:38.471423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:38.499544 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf4596-5c18-430e-bcc6-42d9bc190f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:38.500416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf4596-5c18-430e-bcc6-42d9bc190f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:38.500702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf4596-5c18-430e-bcc6-42d9bc190f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:38.500984 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf4596-5c18-430e-bcc6-42d9bc190f4b admin admin] PATH_INFO: `/v3/projects/3b28f44d18ed4241b89d63e9e28e1c3a` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:38.502195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf4596-5c18-430e-bcc6-42d9bc190f4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b28f44d18ed4241b89d63e9e28e1c3a)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:38.504241 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf4596-5c18-430e-bcc6-42d9bc190f4b admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:38.554594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ecdf4596-5c18-430e-bcc6-42d9bc190f4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b28f44d18ed4241b89d63e9e28e1c3a', 'request_id': 'req-ecdf4596-5c18-430e-bcc6-42d9bc190f4b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:38.562282 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ecdf4596-5c18-430e-bcc6-42d9bc190f4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b28f44d18ed4241b89d63e9e28e1c3a', 'request_id': 'req-ecdf4596-5c18-430e-bcc6-42d9bc190f4b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:38.573659 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ecdf4596-5c18-430e-bcc6-42d9bc190f4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b28f44d18ed4241b89d63e9e28e1c3a', 'request_id': 'req-ecdf4596-5c18-430e-bcc6-42d9bc190f4b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:38.585709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ecdf4596-5c18-430e-bcc6-42d9bc190f4b admin admin] The token cache is being invalidate because project 3b28f44d18ed4241b89d63e9e28e1c3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:37:38.586204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ecdf4596-5c18-430e-bcc6-42d9bc190f4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:38.586886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ecdf4596-5c18-430e-bcc6-42d9bc190f4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:38.588786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1028/2039] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:37:38 2022] DELETE /identity/v3/projects/3b28f44d18ed4241b89d63e9e28e1c3a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:38.594609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9529b3-a172-467a-9c24-488f174447b6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:38.595139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9529b3-a172-467a-9c24-488f174447b6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:38.595228 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9529b3-a172-467a-9c24-488f174447b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:38.617199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6a9529b3-a172-467a-9c24-488f174447b6 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:38.618418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6a9529b3-a172-467a-9c24-488f174447b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:38.679650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:38.698965 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1012/2040] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:38.707691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e989a28-1406-459e-a0b6-d22db02073a4 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:38.708463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e989a28-1406-459e-a0b6-d22db02073a4 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:38.708888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e989a28-1406-459e-a0b6-d22db02073a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:38.729788 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-9e989a28-1406-459e-a0b6-d22db02073a4 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:38.731388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-9e989a28-1406-459e-a0b6-d22db02073a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:38.744467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1029/2041] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:37:38.749149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9529b3-a172-467a-9c24-488f174447b6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:38.760147 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9529b3-a172-467a-9c24-488f174447b6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:38.770824 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce007044-de04-4093-ae6f-400510a681b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:38.771475 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce007044-de04-4093-ae6f-400510a681b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:38.771912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce007044-de04-4093-ae6f-400510a681b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:38.772209 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce007044-de04-4093-ae6f-400510a681b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:38.773152 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce007044-de04-4093-ae6f-400510a681b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:37:38.773750 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce007044-de04-4093-ae6f-400510a681b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:38.776335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce007044-de04-4093-ae6f-400510a681b0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:38.795225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1013/2042] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:37:38.799459 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e989a28-1406-459e-a0b6-d22db02073a4 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:38.809502 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e989a28-1406-459e-a0b6-d22db02073a4 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:38.814164 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689dd5c-eabb-43ec-a26a-2577998eccab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:38.814847 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689dd5c-eabb-43ec-a26a-2577998eccab admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:38.815160 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689dd5c-eabb-43ec-a26a-2577998eccab admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:38.815441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689dd5c-eabb-43ec-a26a-2577998eccab admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:38.816136 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8689dd5c-eabb-43ec-a26a-2577998eccab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:38.818081 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8689dd5c-eabb-43ec-a26a-2577998eccab admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:38.851761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1030/2043] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:37:38 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:37:38.855481 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce007044-de04-4093-ae6f-400510a681b0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:38.866658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce007044-de04-4093-ae6f-400510a681b0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:38.930339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10906238-dd01-40cf-b882-ae01c848d410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:38.931060 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10906238-dd01-40cf-b882-ae01c848d410 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:38.931357 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10906238-dd01-40cf-b882-ae01c848d410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:38.931643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10906238-dd01-40cf-b882-ae01c848d410 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:38.932351 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10906238-dd01-40cf-b882-ae01c848d410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:38.934447 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10906238-dd01-40cf-b882-ae01c848d410 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:38.962800 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1014/2044] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:37:38 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:38.964150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689dd5c-eabb-43ec-a26a-2577998eccab admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:38.975712 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689dd5c-eabb-43ec-a26a-2577998eccab admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:38.980569 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746f907-d710-4d1e-b7c5-b31011b872a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:38.981349 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746f907-d710-4d1e-b7c5-b31011b872a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:38.981641 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746f907-d710-4d1e-b7c5-b31011b872a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:38.981940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746f907-d710-4d1e-b7c5-b31011b872a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:38.982683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746f907-d710-4d1e-b7c5-b31011b872a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:38.984353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746f907-d710-4d1e-b7c5-b31011b872a0 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:38.998864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1031/2045] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:39.001748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10906238-dd01-40cf-b882-ae01c848d410 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:39.016101 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10906238-dd01-40cf-b882-ae01c848d410 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:39.021221 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d32558-c070-40e8-b74e-ba6a5b4e2bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:39.022028 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d32558-c070-40e8-b74e-ba6a5b4e2bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:39.022393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d32558-c070-40e8-b74e-ba6a5b4e2bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:39.022750 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d32558-c070-40e8-b74e-ba6a5b4e2bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:39.023589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d32558-c070-40e8-b74e-ba6a5b4e2bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:39.025291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d32558-c070-40e8-b74e-ba6a5b4e2bd2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:39.039471 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1015/2046] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:39.041085 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746f907-d710-4d1e-b7c5-b31011b872a0 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:39.053145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746f907-d710-4d1e-b7c5-b31011b872a0 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:39.058162 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0018b1-36ef-4d16-a9f7-f5ba35f8754f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:39.059759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0018b1-36ef-4d16-a9f7-f5ba35f8754f admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:39.060046 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0018b1-36ef-4d16-a9f7-f5ba35f8754f admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:39.060332 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0018b1-36ef-4d16-a9f7-f5ba35f8754f admin admin] PATH_INFO: `/v3/projects/fb8d0c62a37d4e9b91893127feb451b4/users/c5b504fe84f046ba90a9d91b4b45b8cd/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:39.078160 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0018b1-36ef-4d16-a9f7-f5ba35f8754f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb8d0c62a37d4e9b91893127feb451b4, user_id=c5b504fe84f046ba90a9d91b4b45b8cd, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:39.080935 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0018b1-36ef-4d16-a9f7-f5ba35f8754f admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:39.098154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1032/2047] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:39 2022] PUT /identity/v3/projects/fb8d0c62a37d4e9b91893127feb451b4/users/c5b504fe84f046ba90a9d91b4b45b8cd/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:39.109547 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f25470-3801-4304-9eab-a3037ef587f9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:39.109852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f25470-3801-4304-9eab-a3037ef587f9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:39.110142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f25470-3801-4304-9eab-a3037ef587f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:39.141450 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-93f25470-3801-4304-9eab-a3037ef587f9 None None] MFA Rules not processed for user `c5b504fe84f046ba90a9d91b4b45b8cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:39.142681 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-93f25470-3801-4304-9eab-a3037ef587f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:39.213414 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1016/2048] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:39 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:39.217963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0018b1-36ef-4d16-a9f7-f5ba35f8754f admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:39.287259 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0018b1-36ef-4d16-a9f7-f5ba35f8754f admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:39.292449 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ce99d-b95b-42ee-ba71-cf9640264d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:39.293740 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ce99d-b95b-42ee-ba71-cf9640264d7e admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:39.294124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ce99d-b95b-42ee-ba71-cf9640264d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:39.294506 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ce99d-b95b-42ee-ba71-cf9640264d7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:39.295485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ce99d-b95b-42ee-ba71-cf9640264d7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:39.300648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ce99d-b95b-42ee-ba71-cf9640264d7e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:39.315563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:39.349800 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1033/2049] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:37:39 2022] POST /identity/v3/projects => generated 367 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:37:39.354194 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f25470-3801-4304-9eab-a3037ef587f9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:39.369389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f25470-3801-4304-9eab-a3037ef587f9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:39.453173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:39.470105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb8ea0-fe38-41cc-85c4-4f8ded15de13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:39.470891 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8ea0-fe38-41cc-85c4-4f8ded15de13 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:39.471219 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8ea0-fe38-41cc-85c4-4f8ded15de13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:39.471510 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb8ea0-fe38-41cc-85c4-4f8ded15de13 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:39.473629 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb8ea0-fe38-41cc-85c4-4f8ded15de13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:39.475982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb8ea0-fe38-41cc-85c4-4f8ded15de13 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:39.521172 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1017/2050] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:37:39 2022] POST /identity/v3/users => generated 290 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:37:39.522411 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ce99d-b95b-42ee-ba71-cf9640264d7e admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:39.533502 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ce99d-b95b-42ee-ba71-cf9640264d7e admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:39.538711 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a64a44-fc7c-4f05-b62c-1d6fefa2ef29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:39.539406 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a64a44-fc7c-4f05-b62c-1d6fefa2ef29 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:39.539701 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a64a44-fc7c-4f05-b62c-1d6fefa2ef29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:39.540026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a64a44-fc7c-4f05-b62c-1d6fefa2ef29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:39.540764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a64a44-fc7c-4f05-b62c-1d6fefa2ef29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:39.542800 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a64a44-fc7c-4f05-b62c-1d6fefa2ef29 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:39.554519 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1034/2051] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:39.560007 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb8ea0-fe38-41cc-85c4-4f8ded15de13 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:39.571601 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb8ea0-fe38-41cc-85c4-4f8ded15de13 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:39.577249 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a62668-b9f5-4ee0-94c7-33c573fdb904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:39.578169 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a62668-b9f5-4ee0-94c7-33c573fdb904 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:39.578442 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a62668-b9f5-4ee0-94c7-33c573fdb904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:39.578728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a62668-b9f5-4ee0-94c7-33c573fdb904 admin admin] PATH_INFO: `/v3/projects/7e9be4a96ace467fbd03d7e812b4344b/users/8a7ba35fc9bf4841956598108251a330/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:39.595090 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a62668-b9f5-4ee0-94c7-33c573fdb904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e9be4a96ace467fbd03d7e812b4344b, user_id=8a7ba35fc9bf4841956598108251a330, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:39.600671 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a62668-b9f5-4ee0-94c7-33c573fdb904 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:39.614073 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1018/2052] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:39 2022] PUT /identity/v3/projects/7e9be4a96ace467fbd03d7e812b4344b/users/8a7ba35fc9bf4841956598108251a330/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:39.618304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a64a44-fc7c-4f05-b62c-1d6fefa2ef29 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:39.629335 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a64a44-fc7c-4f05-b62c-1d6fefa2ef29 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:39.696286 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef5633d-198a-443e-b936-0f7ee9fc264c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:39.696971 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5633d-198a-443e-b936-0f7ee9fc264c admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:39.697258 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5633d-198a-443e-b936-0f7ee9fc264c admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:39.697542 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef5633d-198a-443e-b936-0f7ee9fc264c admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:39.698311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5633d-198a-443e-b936-0f7ee9fc264c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:39.700078 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef5633d-198a-443e-b936-0f7ee9fc264c admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:39.710414 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1035/2053] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:37:39.714582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a62668-b9f5-4ee0-94c7-33c573fdb904 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:39.726163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a62668-b9f5-4ee0-94c7-33c573fdb904 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:39.730805 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b252a47-7af9-4896-a0d8-5717541ea66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:39.732323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b252a47-7af9-4896-a0d8-5717541ea66f admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:39.732642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b252a47-7af9-4896-a0d8-5717541ea66f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:39.732927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b252a47-7af9-4896-a0d8-5717541ea66f admin admin] PATH_INFO: `/v3/projects/7e9be4a96ace467fbd03d7e812b4344b/users/8a7ba35fc9bf4841956598108251a330/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:39.734902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b252a47-7af9-4896-a0d8-5717541ea66f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e9be4a96ace467fbd03d7e812b4344b, user_id=8a7ba35fc9bf4841956598108251a330, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:39.737604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b252a47-7af9-4896-a0d8-5717541ea66f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:39.748797 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1019/2054] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:39 2022] PUT /identity/v3/projects/7e9be4a96ace467fbd03d7e812b4344b/users/8a7ba35fc9bf4841956598108251a330/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:39.759360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbaf0e-9def-4f62-a7fb-2f093a4b5265 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:39.759658 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbaf0e-9def-4f62-a7fb-2f093a4b5265 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:39.759951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbaf0e-9def-4f62-a7fb-2f093a4b5265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:39.788475 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-b5cbaf0e-9def-4f62-a7fb-2f093a4b5265 None None] MFA Rules not processed for user `8a7ba35fc9bf4841956598108251a330`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:39.789946 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b5cbaf0e-9def-4f62-a7fb-2f093a4b5265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:39.878608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1036/2055] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:39 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:39.890155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b252a47-7af9-4896-a0d8-5717541ea66f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:39.892131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9b252a47-7af9-4896-a0d8-5717541ea66f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:39.959405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b252a47-7af9-4896-a0d8-5717541ea66f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:39.963087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a5676-cb2a-4770-9e7c-2e297daa4813 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:39.963749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a5676-cb2a-4770-9e7c-2e297daa4813 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:39.964040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a5676-cb2a-4770-9e7c-2e297daa4813 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:39.964322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a5676-cb2a-4770-9e7c-2e297daa4813 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:39.975592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a5676-cb2a-4770-9e7c-2e297daa4813 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:39.977076 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a5676-cb2a-4770-9e7c-2e297daa4813 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:39.987458 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1020/2056] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:39 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:40.036765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbaf0e-9def-4f62-a7fb-2f093a4b5265 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:40.048100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbaf0e-9def-4f62-a7fb-2f093a4b5265 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:40.051993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bb83a-8b91-469d-a956-d6de88679962 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:40.052753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bb83a-8b91-469d-a956-d6de88679962 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:40.053120 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bb83a-8b91-469d-a956-d6de88679962 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:40.053465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377bb83a-8b91-469d-a956-d6de88679962 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:40.119852 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377bb83a-8b91-469d-a956-d6de88679962 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:40.120241 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377bb83a-8b91-469d-a956-d6de88679962 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:40.126346 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:40.135851 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1037/2057] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:40 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:37:41.719676 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e846a-3d50-4703-bf68-f2960de43969 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:41.720178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e846a-3d50-4703-bf68-f2960de43969 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:41.720287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e846a-3d50-4703-bf68-f2960de43969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:41.740167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cf4e846a-3d50-4703-bf68-f2960de43969 None None] MFA Rules not processed for user `bbe630f5c7c243588c5bd33023775d69`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:41.741383 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cf4e846a-3d50-4703-bf68-f2960de43969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:41.807430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1021/2058] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:43.724659 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bb83a-8b91-469d-a956-d6de88679962 service cinder] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:43.790617 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377bb83a-8b91-469d-a956-d6de88679962 service cinder] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:43.795954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de55e8-41a7-4b4e-a0cd-140b61527907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:43.796850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de55e8-41a7-4b4e-a0cd-140b61527907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:43.797135 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de55e8-41a7-4b4e-a0cd-140b61527907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:43.797414 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de55e8-41a7-4b4e-a0cd-140b61527907 admin admin] PATH_INFO: `/v3/users/c5b504fe84f046ba90a9d91b4b45b8cd` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:43.798598 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de55e8-41a7-4b4e-a0cd-140b61527907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5b504fe84f046ba90a9d91b4b45b8cd)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:43.800538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de55e8-41a7-4b4e-a0cd-140b61527907 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:43.847361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-86de55e8-41a7-4b4e-a0cd-140b61527907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5b504fe84f046ba90a9d91b4b45b8cd'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:43.863283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-86de55e8-41a7-4b4e-a0cd-140b61527907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5b504fe84f046ba90a9d91b4b45b8cd'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:43.873416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-86de55e8-41a7-4b4e-a0cd-140b61527907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5b504fe84f046ba90a9d91b4b45b8cd'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:43.881965 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-86de55e8-41a7-4b4e-a0cd-140b61527907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5b504fe84f046ba90a9d91b4b45b8cd'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:43.883246 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-86de55e8-41a7-4b4e-a0cd-140b61527907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5b504fe84f046ba90a9d91b4b45b8cd'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:43.891962 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1038/2059] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:37:43 2022] DELETE /identity/v3/users/c5b504fe84f046ba90a9d91b4b45b8cd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:43.896498 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51bd7c-dfa9-4ff7-88ef-a110397159a4 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:43.897378 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51bd7c-dfa9-4ff7-88ef-a110397159a4 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:43.897850 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51bd7c-dfa9-4ff7-88ef-a110397159a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:43.921854 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cc51bd7c-dfa9-4ff7-88ef-a110397159a4 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:43.923724 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cc51bd7c-dfa9-4ff7-88ef-a110397159a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:43.987531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:44.001296 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1022/2060] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:44.011606 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de55e8-41a7-4b4e-a0cd-140b61527907 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:44.014154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-86de55e8-41a7-4b4e-a0cd-140b61527907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:44.076280 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de55e8-41a7-4b4e-a0cd-140b61527907 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:44.079982 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8366a45-7757-409a-8171-eb98634a17b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:44.080634 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8366a45-7757-409a-8171-eb98634a17b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:44.080953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8366a45-7757-409a-8171-eb98634a17b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:44.081240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8366a45-7757-409a-8171-eb98634a17b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:44.095358 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8366a45-7757-409a-8171-eb98634a17b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:44.096809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8366a45-7757-409a-8171-eb98634a17b9 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:44.106853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1039/2061] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:44.388470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51bd7c-dfa9-4ff7-88ef-a110397159a4 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:44.390109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cc51bd7c-dfa9-4ff7-88ef-a110397159a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:44.404007 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51bd7c-dfa9-4ff7-88ef-a110397159a4 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:44.409748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fba504-d008-42df-894f-7c401fd01d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:44.410646 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fba504-d008-42df-894f-7c401fd01d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:44.410951 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fba504-d008-42df-894f-7c401fd01d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:44.411255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fba504-d008-42df-894f-7c401fd01d01 admin admin] PATH_INFO: `/v3/projects/fb8d0c62a37d4e9b91893127feb451b4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:44.412518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fba504-d008-42df-894f-7c401fd01d01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb8d0c62a37d4e9b91893127feb451b4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:44.414383 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fba504-d008-42df-894f-7c401fd01d01 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:44.464403 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-24fba504-d008-42df-894f-7c401fd01d01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb8d0c62a37d4e9b91893127feb451b4', 'request_id': 'req-24fba504-d008-42df-894f-7c401fd01d01'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:44.472314 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-24fba504-d008-42df-894f-7c401fd01d01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb8d0c62a37d4e9b91893127feb451b4', 'request_id': 'req-24fba504-d008-42df-894f-7c401fd01d01'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:44.478199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-24fba504-d008-42df-894f-7c401fd01d01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb8d0c62a37d4e9b91893127feb451b4', 'request_id': 'req-24fba504-d008-42df-894f-7c401fd01d01'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:44.489718 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-24fba504-d008-42df-894f-7c401fd01d01 admin admin] The token cache is being invalidate because project fb8d0c62a37d4e9b91893127feb451b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:37:44.490007 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-24fba504-d008-42df-894f-7c401fd01d01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:44.490629 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-24fba504-d008-42df-894f-7c401fd01d01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:44.492355 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1023/2062] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:37:44 2022] DELETE /identity/v3/projects/fb8d0c62a37d4e9b91893127feb451b4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:44.497017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8366a45-7757-409a-8171-eb98634a17b9 service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:44.506945 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8366a45-7757-409a-8171-eb98634a17b9 service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:44.522950 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:44.566276 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb9213-2637-4ff2-ae24-3f8e335af414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:44.567194 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb9213-2637-4ff2-ae24-3f8e335af414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:44.567558 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb9213-2637-4ff2-ae24-3f8e335af414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:44.567890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb9213-2637-4ff2-ae24-3f8e335af414 admin admin] PATH_INFO: `/v3/users/8a7ba35fc9bf4841956598108251a330` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:44.569170 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb9213-2637-4ff2-ae24-3f8e335af414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a7ba35fc9bf4841956598108251a330)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:44.571133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb9213-2637-4ff2-ae24-3f8e335af414 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:44.619140 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-6bcb9213-2637-4ff2-ae24-3f8e335af414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a7ba35fc9bf4841956598108251a330'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:44.631976 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-6bcb9213-2637-4ff2-ae24-3f8e335af414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a7ba35fc9bf4841956598108251a330'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:44.640639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-6bcb9213-2637-4ff2-ae24-3f8e335af414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a7ba35fc9bf4841956598108251a330'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:44.649833 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-6bcb9213-2637-4ff2-ae24-3f8e335af414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a7ba35fc9bf4841956598108251a330'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:44.650504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-6bcb9213-2637-4ff2-ae24-3f8e335af414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a7ba35fc9bf4841956598108251a330'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:44.657918 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1040/2063] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:37:44 2022] DELETE /identity/v3/users/8a7ba35fc9bf4841956598108251a330 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:44.942136 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fba504-d008-42df-894f-7c401fd01d01 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:44.944051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-24fba504-d008-42df-894f-7c401fd01d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:44.973040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fba504-d008-42df-894f-7c401fd01d01 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:45.039245 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901700f-ed70-4a77-9091-55da019cdc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:45.040227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901700f-ed70-4a77-9091-55da019cdc19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:45.040590 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901700f-ed70-4a77-9091-55da019cdc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:45.040941 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901700f-ed70-4a77-9091-55da019cdc19 admin admin] PATH_INFO: `/v3/projects/7e9be4a96ace467fbd03d7e812b4344b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:45.042969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901700f-ed70-4a77-9091-55da019cdc19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e9be4a96ace467fbd03d7e812b4344b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:45.045880 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901700f-ed70-4a77-9091-55da019cdc19 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:45.113975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7901700f-ed70-4a77-9091-55da019cdc19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e9be4a96ace467fbd03d7e812b4344b', 'request_id': 'req-7901700f-ed70-4a77-9091-55da019cdc19'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:45.136396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7901700f-ed70-4a77-9091-55da019cdc19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e9be4a96ace467fbd03d7e812b4344b', 'request_id': 'req-7901700f-ed70-4a77-9091-55da019cdc19'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:45.150135 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7901700f-ed70-4a77-9091-55da019cdc19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e9be4a96ace467fbd03d7e812b4344b', 'request_id': 'req-7901700f-ed70-4a77-9091-55da019cdc19'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:45.156282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7901700f-ed70-4a77-9091-55da019cdc19 admin admin] The token cache is being invalidate because project 7e9be4a96ace467fbd03d7e812b4344b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:37:45.156581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7901700f-ed70-4a77-9091-55da019cdc19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:45.157228 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7901700f-ed70-4a77-9091-55da019cdc19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:45.158951 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1024/2064] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:37:44 2022] DELETE /identity/v3/projects/7e9be4a96ace467fbd03d7e812b4344b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:45.163853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec9214-d429-4bc1-b709-bcf11863e8ea None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:45.164336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec9214-d429-4bc1-b709-bcf11863e8ea None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:45.164630 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec9214-d429-4bc1-b709-bcf11863e8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:45.184880 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-0fec9214-d429-4bc1-b709-bcf11863e8ea None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:45.186276 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-0fec9214-d429-4bc1-b709-bcf11863e8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:45.248392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1041/2065] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:45.256130 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297056c-6af8-4e86-9eb5-4fb8bc312878 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:45.257020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297056c-6af8-4e86-9eb5-4fb8bc312878 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:45.257330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297056c-6af8-4e86-9eb5-4fb8bc312878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:45.266391 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:45.278836 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6297056c-6af8-4e86-9eb5-4fb8bc312878 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:45.280147 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6297056c-6af8-4e86-9eb5-4fb8bc312878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:45.292866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1025/2066] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:45.294339 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec9214-d429-4bc1-b709-bcf11863e8ea None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:45.304350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec9214-d429-4bc1-b709-bcf11863e8ea None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:45.309843 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d7d35-21ee-4c82-81d6-3acaf947a3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:45.310544 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d7d35-21ee-4c82-81d6-3acaf947a3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:45.310878 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d7d35-21ee-4c82-81d6-3acaf947a3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:45.311234 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d7d35-21ee-4c82-81d6-3acaf947a3f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:45.312071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d7d35-21ee-4c82-81d6-3acaf947a3f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:37:45.312378 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d7d35-21ee-4c82-81d6-3acaf947a3f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:45.313961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d7d35-21ee-4c82-81d6-3acaf947a3f0 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:45.326958 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1042/2067] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:37:45.331027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297056c-6af8-4e86-9eb5-4fb8bc312878 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:45.342510 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297056c-6af8-4e86-9eb5-4fb8bc312878 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:45.347414 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a43f86-20cb-46d7-8f2e-9fd8c2fcbec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:45.348071 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a43f86-20cb-46d7-8f2e-9fd8c2fcbec2 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:45.348364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a43f86-20cb-46d7-8f2e-9fd8c2fcbec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:45.348650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a43f86-20cb-46d7-8f2e-9fd8c2fcbec2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:45.349419 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a43f86-20cb-46d7-8f2e-9fd8c2fcbec2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:45.351413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a43f86-20cb-46d7-8f2e-9fd8c2fcbec2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:45.379786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1026/2068] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:37:45 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:45.384969 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d7d35-21ee-4c82-81d6-3acaf947a3f0 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:45.395743 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d7d35-21ee-4c82-81d6-3acaf947a3f0 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:45.422111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:45.462028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7a340-3876-49a6-804d-c692f0c4a639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:45.462975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7a340-3876-49a6-804d-c692f0c4a639 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:45.463947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7a340-3876-49a6-804d-c692f0c4a639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:45.464312 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7a340-3876-49a6-804d-c692f0c4a639 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:45.466266 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7a340-3876-49a6-804d-c692f0c4a639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:45.468422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7a340-3876-49a6-804d-c692f0c4a639 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:45.490946 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1043/2069] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:37:45 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:45.496054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a43f86-20cb-46d7-8f2e-9fd8c2fcbec2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:45.508314 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a43f86-20cb-46d7-8f2e-9fd8c2fcbec2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:45.513743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75d66d-6746-4823-9611-bdce8c876b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:45.514417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75d66d-6746-4823-9611-bdce8c876b16 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:45.514709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75d66d-6746-4823-9611-bdce8c876b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:45.515032 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75d66d-6746-4823-9611-bdce8c876b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:45.515776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75d66d-6746-4823-9611-bdce8c876b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:45.517472 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75d66d-6746-4823-9611-bdce8c876b16 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:45.529686 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1027/2070] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:45.534944 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7a340-3876-49a6-804d-c692f0c4a639 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:45.553921 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7a340-3876-49a6-804d-c692f0c4a639 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:45.566135 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbd1a2-40c5-43a5-b7ec-ae09075853f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:45.567924 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbd1a2-40c5-43a5-b7ec-ae09075853f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:45.568095 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbd1a2-40c5-43a5-b7ec-ae09075853f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:45.568587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbd1a2-40c5-43a5-b7ec-ae09075853f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:45.569365 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbd1a2-40c5-43a5-b7ec-ae09075853f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:45.571166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbd1a2-40c5-43a5-b7ec-ae09075853f7 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:45.591682 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1044/2071] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:45.595937 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75d66d-6746-4823-9611-bdce8c876b16 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:45.618356 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75d66d-6746-4823-9611-bdce8c876b16 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:45.624796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34097783-584e-47f4-b8d9-7e5ca546aaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:45.625830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34097783-584e-47f4-b8d9-7e5ca546aaaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:45.626685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34097783-584e-47f4-b8d9-7e5ca546aaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:45.626685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34097783-584e-47f4-b8d9-7e5ca546aaaf admin admin] PATH_INFO: `/v3/projects/635f6d79f89b402683a60e2158527a07/users/db76b74505fc486980b6c8dd6c7cb3ba/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:45.651144 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34097783-584e-47f4-b8d9-7e5ca546aaaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=635f6d79f89b402683a60e2158527a07, user_id=db76b74505fc486980b6c8dd6c7cb3ba, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:45.655698 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34097783-584e-47f4-b8d9-7e5ca546aaaf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:45.677298 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1028/2072] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:45 2022] PUT /identity/v3/projects/635f6d79f89b402683a60e2158527a07/users/db76b74505fc486980b6c8dd6c7cb3ba/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:45.690039 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb757d8c-147f-49a6-a1c0-cb36ef8cbaf9 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:45.690379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb757d8c-147f-49a6-a1c0-cb36ef8cbaf9 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:45.690610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb757d8c-147f-49a6-a1c0-cb36ef8cbaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:45.754889 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-eb757d8c-147f-49a6-a1c0-cb36ef8cbaf9 None None] MFA Rules not processed for user `db76b74505fc486980b6c8dd6c7cb3ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:45.756514 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-eb757d8c-147f-49a6-a1c0-cb36ef8cbaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:45.846302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1045/2073] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:45.850552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34097783-584e-47f4-b8d9-7e5ca546aaaf admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:45.922024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34097783-584e-47f4-b8d9-7e5ca546aaaf admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:45.926726 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11262fb8-e3e0-4263-aa95-f32b4e37150a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:45.927421 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11262fb8-e3e0-4263-aa95-f32b4e37150a admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:45.927708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11262fb8-e3e0-4263-aa95-f32b4e37150a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:45.928045 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11262fb8-e3e0-4263-aa95-f32b4e37150a admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:45.928755 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11262fb8-e3e0-4263-aa95-f32b4e37150a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:45.930644 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11262fb8-e3e0-4263-aa95-f32b4e37150a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:45.962443 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1029/2074] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:37:45 2022] POST /identity/v3/projects => generated 393 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:37:45.964532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb757d8c-147f-49a6-a1c0-cb36ef8cbaf9 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:45.977636 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb757d8c-147f-49a6-a1c0-cb36ef8cbaf9 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:46.045174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820e859-a45b-4d78-b4c2-b8ff533d745d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:46.045786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820e859-a45b-4d78-b4c2-b8ff533d745d admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:46.046111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820e859-a45b-4d78-b4c2-b8ff533d745d admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:46.046392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820e859-a45b-4d78-b4c2-b8ff533d745d admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:46.047236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820e859-a45b-4d78-b4c2-b8ff533d745d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:46.049088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820e859-a45b-4d78-b4c2-b8ff533d745d admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:46.069338 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1046/2075] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:37:45 2022] POST /identity/v3/users => generated 303 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:37:46.073300 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11262fb8-e3e0-4263-aa95-f32b4e37150a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:46.084269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11262fb8-e3e0-4263-aa95-f32b4e37150a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:46.089330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efde8de-6cee-44d8-94e5-a2b50ef6974d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:46.090003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efde8de-6cee-44d8-94e5-a2b50ef6974d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:46.090886 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efde8de-6cee-44d8-94e5-a2b50ef6974d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:46.091380 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efde8de-6cee-44d8-94e5-a2b50ef6974d admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:46.092122 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efde8de-6cee-44d8-94e5-a2b50ef6974d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:46.093724 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efde8de-6cee-44d8-94e5-a2b50ef6974d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:46.104604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1030/2076] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:46.109126 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820e859-a45b-4d78-b4c2-b8ff533d745d admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:46.120638 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820e859-a45b-4d78-b4c2-b8ff533d745d admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:46.128458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff20bce-1132-4af9-a967-872bca6a9a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:46.129468 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff20bce-1132-4af9-a967-872bca6a9a15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:46.129758 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff20bce-1132-4af9-a967-872bca6a9a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:46.130038 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff20bce-1132-4af9-a967-872bca6a9a15 admin admin] PATH_INFO: `/v3/projects/02fa3ae0275e4f4fbea80ea3a3650ced/users/6d954a6226b4452f821d5d4f7f53f90b/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:46.162271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff20bce-1132-4af9-a967-872bca6a9a15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02fa3ae0275e4f4fbea80ea3a3650ced, user_id=6d954a6226b4452f821d5d4f7f53f90b, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:46.164530 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff20bce-1132-4af9-a967-872bca6a9a15 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:46.177082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:46.186264 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1047/2077] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:46 2022] PUT /identity/v3/projects/02fa3ae0275e4f4fbea80ea3a3650ced/users/6d954a6226b4452f821d5d4f7f53f90b/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:46.190208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efde8de-6cee-44d8-94e5-a2b50ef6974d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:46.201785 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efde8de-6cee-44d8-94e5-a2b50ef6974d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:46.275883 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9719e47-613b-43c0-89a9-64cf30f6bc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:46.276585 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9719e47-613b-43c0-89a9-64cf30f6bc37 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:46.276926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9719e47-613b-43c0-89a9-64cf30f6bc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:46.277208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9719e47-613b-43c0-89a9-64cf30f6bc37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:46.278200 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9719e47-613b-43c0-89a9-64cf30f6bc37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:46.280487 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9719e47-613b-43c0-89a9-64cf30f6bc37 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:46.293653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1031/2078] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:37:46.297721 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff20bce-1132-4af9-a967-872bca6a9a15 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:46.309821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff20bce-1132-4af9-a967-872bca6a9a15 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:46.314628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c18fdf-a577-4999-b841-8ac3ffc21e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:46.315554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c18fdf-a577-4999-b841-8ac3ffc21e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:46.315839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c18fdf-a577-4999-b841-8ac3ffc21e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:46.316113 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c18fdf-a577-4999-b841-8ac3ffc21e0e admin admin] PATH_INFO: `/v3/projects/02fa3ae0275e4f4fbea80ea3a3650ced/users/6d954a6226b4452f821d5d4f7f53f90b/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:46.318080 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c18fdf-a577-4999-b841-8ac3ffc21e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=02fa3ae0275e4f4fbea80ea3a3650ced, user_id=6d954a6226b4452f821d5d4f7f53f90b, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:46.320767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c18fdf-a577-4999-b841-8ac3ffc21e0e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:46.332250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1048/2079] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:46 2022] PUT /identity/v3/projects/02fa3ae0275e4f4fbea80ea3a3650ced/users/6d954a6226b4452f821d5d4f7f53f90b/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:46.342642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102668cc-370f-4116-8ab9-86f28be946a6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:46.343049 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102668cc-370f-4116-8ab9-86f28be946a6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:46.343804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102668cc-370f-4116-8ab9-86f28be946a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:46.364118 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:46.373274 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-102668cc-370f-4116-8ab9-86f28be946a6 None None] MFA Rules not processed for user `6d954a6226b4452f821d5d4f7f53f90b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:46.375201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-102668cc-370f-4116-8ab9-86f28be946a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:46.456269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1032/2080] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:46.468655 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c18fdf-a577-4999-b841-8ac3ffc21e0e admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:46.470769 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-76c18fdf-a577-4999-b841-8ac3ffc21e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:46.531507 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c18fdf-a577-4999-b841-8ac3ffc21e0e admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:46.536037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5baf6d1-8980-47f8-8744-f272e1c4b24a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:46.536701 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5baf6d1-8980-47f8-8744-f272e1c4b24a service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:46.536988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5baf6d1-8980-47f8-8744-f272e1c4b24a service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:46.537267 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5baf6d1-8980-47f8-8744-f272e1c4b24a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:46.547398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5baf6d1-8980-47f8-8744-f272e1c4b24a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:46.549225 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5baf6d1-8980-47f8-8744-f272e1c4b24a service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:46.559334 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1049/2081] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:46.608878 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102668cc-370f-4116-8ab9-86f28be946a6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:46.610294 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-102668cc-370f-4116-8ab9-86f28be946a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:46.622195 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102668cc-370f-4116-8ab9-86f28be946a6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:46.625920 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a29b5-e0b6-423c-b1c4-3500136f4da0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:46.626572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a29b5-e0b6-423c-b1c4-3500136f4da0 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:46.626863 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a29b5-e0b6-423c-b1c4-3500136f4da0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:46.627183 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a29b5-e0b6-423c-b1c4-3500136f4da0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:46.687317 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a29b5-e0b6-423c-b1c4-3500136f4da0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:46.688912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a29b5-e0b6-423c-b1c4-3500136f4da0 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:46.698093 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1033/2082] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:53.389811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5baf6d1-8980-47f8-8744-f272e1c4b24a service cinder] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:53.471374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5baf6d1-8980-47f8-8744-f272e1c4b24a service cinder] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:53.475852 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff3086-7722-40ea-b701-977c020f21ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:53.476729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff3086-7722-40ea-b701-977c020f21ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:53.477013 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff3086-7722-40ea-b701-977c020f21ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:53.477322 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff3086-7722-40ea-b701-977c020f21ba admin admin] PATH_INFO: `/v3/users/db76b74505fc486980b6c8dd6c7cb3ba` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:53.478504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff3086-7722-40ea-b701-977c020f21ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=db76b74505fc486980b6c8dd6c7cb3ba)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:53.480438 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff3086-7722-40ea-b701-977c020f21ba admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:53.530347 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-99ff3086-7722-40ea-b701-977c020f21ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db76b74505fc486980b6c8dd6c7cb3ba'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:53.543777 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-99ff3086-7722-40ea-b701-977c020f21ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db76b74505fc486980b6c8dd6c7cb3ba'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:53.551417 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-99ff3086-7722-40ea-b701-977c020f21ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db76b74505fc486980b6c8dd6c7cb3ba'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:53.558783 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-99ff3086-7722-40ea-b701-977c020f21ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db76b74505fc486980b6c8dd6c7cb3ba'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:53.560206 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-99ff3086-7722-40ea-b701-977c020f21ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db76b74505fc486980b6c8dd6c7cb3ba'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:53.568191 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1050/2083] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:37:53 2022] DELETE /identity/v3/users/db76b74505fc486980b6c8dd6c7cb3ba => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:53.572584 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdced1-2fb3-4dc9-9d24-ec5b743d9534 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:53.573074 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdced1-2fb3-4dc9-9d24-ec5b743d9534 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:53.573169 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdced1-2fb3-4dc9-9d24-ec5b743d9534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:53.592489 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b4bdced1-2fb3-4dc9-9d24-ec5b743d9534 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:53.593706 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b4bdced1-2fb3-4dc9-9d24-ec5b743d9534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:53.669575 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1034/2084] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:53.683325 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff3086-7722-40ea-b701-977c020f21ba admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:53.685573 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-99ff3086-7722-40ea-b701-977c020f21ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:53.701239 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:53.764425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff3086-7722-40ea-b701-977c020f21ba admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:53.777134 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bc222-43a2-483f-adbd-eb468e2f6145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:53.778525 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bc222-43a2-483f-adbd-eb468e2f6145 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:53.778991 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bc222-43a2-483f-adbd-eb468e2f6145 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:53.779314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bc222-43a2-483f-adbd-eb468e2f6145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:53.791954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bc222-43a2-483f-adbd-eb468e2f6145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:53.794107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bc222-43a2-483f-adbd-eb468e2f6145 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:53.806022 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1051/2085] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:37:54.368895 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdced1-2fb3-4dc9-9d24-ec5b743d9534 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:54.370311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b4bdced1-2fb3-4dc9-9d24-ec5b743d9534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:54.383163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdced1-2fb3-4dc9-9d24-ec5b743d9534 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:54.389325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e2e33-1315-4daa-b65b-3d20d99e8375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:54.390205 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e2e33-1315-4daa-b65b-3d20d99e8375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:54.390493 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e2e33-1315-4daa-b65b-3d20d99e8375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:54.390775 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e2e33-1315-4daa-b65b-3d20d99e8375 admin admin] PATH_INFO: `/v3/projects/635f6d79f89b402683a60e2158527a07` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:54.392021 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e2e33-1315-4daa-b65b-3d20d99e8375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=635f6d79f89b402683a60e2158527a07)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:54.393918 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e2e33-1315-4daa-b65b-3d20d99e8375 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:54.433646 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:54.458709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-191e2e33-1315-4daa-b65b-3d20d99e8375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '635f6d79f89b402683a60e2158527a07', 'request_id': 'req-191e2e33-1315-4daa-b65b-3d20d99e8375'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:54.469737 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-191e2e33-1315-4daa-b65b-3d20d99e8375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '635f6d79f89b402683a60e2158527a07', 'request_id': 'req-191e2e33-1315-4daa-b65b-3d20d99e8375'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:54.477724 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-191e2e33-1315-4daa-b65b-3d20d99e8375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '635f6d79f89b402683a60e2158527a07', 'request_id': 'req-191e2e33-1315-4daa-b65b-3d20d99e8375'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:54.494656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-191e2e33-1315-4daa-b65b-3d20d99e8375 admin admin] The token cache is being invalidate because project 635f6d79f89b402683a60e2158527a07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:37:54.495035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-191e2e33-1315-4daa-b65b-3d20d99e8375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:54.497501 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-191e2e33-1315-4daa-b65b-3d20d99e8375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:54.503401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1035/2086] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:37:54 2022] DELETE /identity/v3/projects/635f6d79f89b402683a60e2158527a07 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:54.507762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bc222-43a2-483f-adbd-eb468e2f6145 service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:54.523015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bc222-43a2-483f-adbd-eb468e2f6145 service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:54.598890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfd09a-7f8e-46f3-9f84-84ae559010a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:54.600460 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfd09a-7f8e-46f3-9f84-84ae559010a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:54.600460 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfd09a-7f8e-46f3-9f84-84ae559010a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:54.600460 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfd09a-7f8e-46f3-9f84-84ae559010a0 admin admin] PATH_INFO: `/v3/users/6d954a6226b4452f821d5d4f7f53f90b` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:54.602153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bfd09a-7f8e-46f3-9f84-84ae559010a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d954a6226b4452f821d5d4f7f53f90b)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:54.604123 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bfd09a-7f8e-46f3-9f84-84ae559010a0 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:54.656128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-62bfd09a-7f8e-46f3-9f84-84ae559010a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d954a6226b4452f821d5d4f7f53f90b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:54.670132 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-62bfd09a-7f8e-46f3-9f84-84ae559010a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d954a6226b4452f821d5d4f7f53f90b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:54.677920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-62bfd09a-7f8e-46f3-9f84-84ae559010a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d954a6226b4452f821d5d4f7f53f90b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:54.686599 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-62bfd09a-7f8e-46f3-9f84-84ae559010a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d954a6226b4452f821d5d4f7f53f90b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:54.686898 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-62bfd09a-7f8e-46f3-9f84-84ae559010a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d954a6226b4452f821d5d4f7f53f90b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:54.693497 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1052/2087] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:37:54 2022] DELETE /identity/v3/users/6d954a6226b4452f821d5d4f7f53f90b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:55.016065 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e2e33-1315-4daa-b65b-3d20d99e8375 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:55.017647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-191e2e33-1315-4daa-b65b-3d20d99e8375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:55.035534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e2e33-1315-4daa-b65b-3d20d99e8375 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:55.109272 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4398ea7-01e9-4cfb-8c55-57604a017010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:55.110143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4398ea7-01e9-4cfb-8c55-57604a017010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:55.110429 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4398ea7-01e9-4cfb-8c55-57604a017010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:55.110706 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4398ea7-01e9-4cfb-8c55-57604a017010 admin admin] PATH_INFO: `/v3/projects/02fa3ae0275e4f4fbea80ea3a3650ced` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:55.111986 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4398ea7-01e9-4cfb-8c55-57604a017010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02fa3ae0275e4f4fbea80ea3a3650ced)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:55.113894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4398ea7-01e9-4cfb-8c55-57604a017010 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:55.167665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f4398ea7-01e9-4cfb-8c55-57604a017010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02fa3ae0275e4f4fbea80ea3a3650ced', 'request_id': 'req-f4398ea7-01e9-4cfb-8c55-57604a017010'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:55.176565 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f4398ea7-01e9-4cfb-8c55-57604a017010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02fa3ae0275e4f4fbea80ea3a3650ced', 'request_id': 'req-f4398ea7-01e9-4cfb-8c55-57604a017010'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:55.182642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f4398ea7-01e9-4cfb-8c55-57604a017010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02fa3ae0275e4f4fbea80ea3a3650ced', 'request_id': 'req-f4398ea7-01e9-4cfb-8c55-57604a017010'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:55.195092 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f4398ea7-01e9-4cfb-8c55-57604a017010 admin admin] The token cache is being invalidate because project 02fa3ae0275e4f4fbea80ea3a3650ced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:37:55.195389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f4398ea7-01e9-4cfb-8c55-57604a017010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:55.196053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f4398ea7-01e9-4cfb-8c55-57604a017010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:55.197605 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1036/2088] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:37:55 2022] DELETE /identity/v3/projects/02fa3ae0275e4f4fbea80ea3a3650ced => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:55.204885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1491a-7d05-4ae1-99c0-c7259be50807 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:55.205185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1491a-7d05-4ae1-99c0-c7259be50807 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:55.205471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1491a-7d05-4ae1-99c0-c7259be50807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:55.225536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-a5b1491a-7d05-4ae1-99c0-c7259be50807 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:55.226763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-a5b1491a-7d05-4ae1-99c0-c7259be50807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:55.270191 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:55.326637 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1053/2089] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:55.337648 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3759a-2827-4c84-a81e-4b51748d7115 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:55.338322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3759a-2827-4c84-a81e-4b51748d7115 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:55.338858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3759a-2827-4c84-a81e-4b51748d7115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:55.383998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-56c3759a-2827-4c84-a81e-4b51748d7115 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:55.385377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-56c3759a-2827-4c84-a81e-4b51748d7115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:55.394478 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1037/2090] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:37:55.398562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1491a-7d05-4ae1-99c0-c7259be50807 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:55.412369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1491a-7d05-4ae1-99c0-c7259be50807 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:55.417052 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558139f0-5229-43d3-a7fe-b8bef971dd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:55.417714 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558139f0-5229-43d3-a7fe-b8bef971dd8f admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:55.418007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558139f0-5229-43d3-a7fe-b8bef971dd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:55.419312 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558139f0-5229-43d3-a7fe-b8bef971dd8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:55.419720 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558139f0-5229-43d3-a7fe-b8bef971dd8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:37:55.420034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558139f0-5229-43d3-a7fe-b8bef971dd8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:55.421645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558139f0-5229-43d3-a7fe-b8bef971dd8f admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:55.434380 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1054/2091] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:37:55.438804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3759a-2827-4c84-a81e-4b51748d7115 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:55.448724 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3759a-2827-4c84-a81e-4b51748d7115 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:55.453585 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36723e6-962f-4fe0-ac0b-d1e3562f5a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:55.454254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36723e6-962f-4fe0-ac0b-d1e3562f5a72 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:55.454546 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36723e6-962f-4fe0-ac0b-d1e3562f5a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:55.454834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36723e6-962f-4fe0-ac0b-d1e3562f5a72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:55.455579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36723e6-962f-4fe0-ac0b-d1e3562f5a72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:55.457480 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36723e6-962f-4fe0-ac0b-d1e3562f5a72 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:55.487879 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1038/2092] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:37:55 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:37:55.489156 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558139f0-5229-43d3-a7fe-b8bef971dd8f admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:55.500419 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558139f0-5229-43d3-a7fe-b8bef971dd8f admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:55.560297 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6321f7d-2234-4c18-a1eb-136e6c3e4239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:55.560979 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6321f7d-2234-4c18-a1eb-136e6c3e4239 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:55.561275 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6321f7d-2234-4c18-a1eb-136e6c3e4239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:55.561560 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6321f7d-2234-4c18-a1eb-136e6c3e4239 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:55.562251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6321f7d-2234-4c18-a1eb-136e6c3e4239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:55.564243 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6321f7d-2234-4c18-a1eb-136e6c3e4239 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:55.590612 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1055/2093] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:37:55 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:55.594462 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36723e6-962f-4fe0-ac0b-d1e3562f5a72 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:55.610611 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36723e6-962f-4fe0-ac0b-d1e3562f5a72 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:55.615525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e6ec7-73fb-4344-91bf-29e26ea7f099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:55.616206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e6ec7-73fb-4344-91bf-29e26ea7f099 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:55.616662 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e6ec7-73fb-4344-91bf-29e26ea7f099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:55.616950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e6ec7-73fb-4344-91bf-29e26ea7f099 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:55.617769 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e6ec7-73fb-4344-91bf-29e26ea7f099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:55.619461 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e6ec7-73fb-4344-91bf-29e26ea7f099 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:55.631563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1039/2094] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:55.635590 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6321f7d-2234-4c18-a1eb-136e6c3e4239 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:55.646881 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6321f7d-2234-4c18-a1eb-136e6c3e4239 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:55.652022 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2ff77-a7f7-499c-97a6-d8c8b4976564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:55.652815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2ff77-a7f7-499c-97a6-d8c8b4976564 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:55.653202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2ff77-a7f7-499c-97a6-d8c8b4976564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:55.653440 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2ff77-a7f7-499c-97a6-d8c8b4976564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:55.654176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2ff77-a7f7-499c-97a6-d8c8b4976564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:55.655822 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2ff77-a7f7-499c-97a6-d8c8b4976564 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:55.666896 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1056/2095] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:55.672236 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e6ec7-73fb-4344-91bf-29e26ea7f099 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:55.686548 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e6ec7-73fb-4344-91bf-29e26ea7f099 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:55.691238 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4a82c-47d8-4d16-8690-ad588f559b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:55.692118 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4a82c-47d8-4d16-8690-ad588f559b79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:55.692402 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4a82c-47d8-4d16-8690-ad588f559b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:55.692682 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4a82c-47d8-4d16-8690-ad588f559b79 admin admin] PATH_INFO: `/v3/projects/449865b12c7b4be0830db895d33ca0ae/users/9959e62a4fee4385ba31455e8e76f76e/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:55.696023 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:55.711261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4a82c-47d8-4d16-8690-ad588f559b79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=449865b12c7b4be0830db895d33ca0ae, user_id=9959e62a4fee4385ba31455e8e76f76e, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:55.714961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4a82c-47d8-4d16-8690-ad588f559b79 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:55.729140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1040/2096] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:55 2022] PUT /identity/v3/projects/449865b12c7b4be0830db895d33ca0ae/users/9959e62a4fee4385ba31455e8e76f76e/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:55.747951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1ad06-54f6-4e6b-baa9-da41767e943d None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:55.748283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1ad06-54f6-4e6b-baa9-da41767e943d None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:55.748575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e1ad06-54f6-4e6b-baa9-da41767e943d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:55.783916 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-64e1ad06-54f6-4e6b-baa9-da41767e943d None None] MFA Rules not processed for user `9959e62a4fee4385ba31455e8e76f76e`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:55.785522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-64e1ad06-54f6-4e6b-baa9-da41767e943d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:55.870942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1057/2097] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:55.876509 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4a82c-47d8-4d16-8690-ad588f559b79 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:55.948472 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4a82c-47d8-4d16-8690-ad588f559b79 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:55.954134 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f9490-fa80-4924-917d-058308d3c93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:55.954677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f9490-fa80-4924-917d-058308d3c93a admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:55.955003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f9490-fa80-4924-917d-058308d3c93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:55.955290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f9490-fa80-4924-917d-058308d3c93a admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:55.955989 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f9490-fa80-4924-917d-058308d3c93a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:55.957883 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f9490-fa80-4924-917d-058308d3c93a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:55.992561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1041/2098] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:37:55 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:55.996532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1ad06-54f6-4e6b-baa9-da41767e943d None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:56.007206 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e1ad06-54f6-4e6b-baa9-da41767e943d None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:56.031279 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:56.065556 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd8634-7a39-45f7-8fa0-b7187057aeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:56.066229 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd8634-7a39-45f7-8fa0-b7187057aeaf admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:56.066525 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd8634-7a39-45f7-8fa0-b7187057aeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:56.067080 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd8634-7a39-45f7-8fa0-b7187057aeaf admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:56.067828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd8634-7a39-45f7-8fa0-b7187057aeaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:56.069875 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd8634-7a39-45f7-8fa0-b7187057aeaf admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:56.097156 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1058/2099] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:37:55 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:56.098474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f9490-fa80-4924-917d-058308d3c93a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:56.110228 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f9490-fa80-4924-917d-058308d3c93a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:56.116120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db849581-02f4-48af-8966-2d29b6a79751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:56.116788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db849581-02f4-48af-8966-2d29b6a79751 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:56.117195 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db849581-02f4-48af-8966-2d29b6a79751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:56.117483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db849581-02f4-48af-8966-2d29b6a79751 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:56.118210 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db849581-02f4-48af-8966-2d29b6a79751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:56.119890 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db849581-02f4-48af-8966-2d29b6a79751 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:56.135470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1042/2100] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:37:56.139248 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd8634-7a39-45f7-8fa0-b7187057aeaf admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:56.149562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd8634-7a39-45f7-8fa0-b7187057aeaf admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:56.154879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52283c-752f-473d-bc87-60dbc54adc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:56.155567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52283c-752f-473d-bc87-60dbc54adc00 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:56.155855 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52283c-752f-473d-bc87-60dbc54adc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:56.156139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52283c-752f-473d-bc87-60dbc54adc00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:56.156870 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52283c-752f-473d-bc87-60dbc54adc00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:56.158470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52283c-752f-473d-bc87-60dbc54adc00 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:56.169649 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1059/2101] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:56.173747 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db849581-02f4-48af-8966-2d29b6a79751 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:56.184928 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db849581-02f4-48af-8966-2d29b6a79751 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:56.190340 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25375395-090f-4c21-8247-845cb9ae1224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:56.191391 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25375395-090f-4c21-8247-845cb9ae1224 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:56.191681 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25375395-090f-4c21-8247-845cb9ae1224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:56.191968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25375395-090f-4c21-8247-845cb9ae1224 admin admin] PATH_INFO: `/v3/projects/863222117d1f4c4aa113ff37ee814a09/users/5f45e6af3786488bbc8d1005a889a99f/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:56.212426 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25375395-090f-4c21-8247-845cb9ae1224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=863222117d1f4c4aa113ff37ee814a09, user_id=5f45e6af3786488bbc8d1005a889a99f, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:56.216047 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25375395-090f-4c21-8247-845cb9ae1224 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:56.230282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1043/2102] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:56 2022] PUT /identity/v3/projects/863222117d1f4c4aa113ff37ee814a09/users/5f45e6af3786488bbc8d1005a889a99f/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:56.241529 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f944d2-128a-44f4-b8a5-94e3a24d8300 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:56.241903 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f944d2-128a-44f4-b8a5-94e3a24d8300 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:56.242199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f944d2-128a-44f4-b8a5-94e3a24d8300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:56.269037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-77f944d2-128a-44f4-b8a5-94e3a24d8300 None None] MFA Rules not processed for user `5f45e6af3786488bbc8d1005a889a99f`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:56.270953 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-77f944d2-128a-44f4-b8a5-94e3a24d8300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:56.342562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1060/2103] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:56.346949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25375395-090f-4c21-8247-845cb9ae1224 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:56.422441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25375395-090f-4c21-8247-845cb9ae1224 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:56.427812 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0a5fc-1e52-437f-9b63-2a800ef0df74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:56.428577 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0a5fc-1e52-437f-9b63-2a800ef0df74 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:56.428869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0a5fc-1e52-437f-9b63-2a800ef0df74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:56.429180 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde0a5fc-1e52-437f-9b63-2a800ef0df74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:56.429888 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0a5fc-1e52-437f-9b63-2a800ef0df74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:56.431864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde0a5fc-1e52-437f-9b63-2a800ef0df74 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:56.459918 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1044/2104] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:37:56 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:56.464999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f944d2-128a-44f4-b8a5-94e3a24d8300 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:56.484262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f944d2-128a-44f4-b8a5-94e3a24d8300 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:56.551059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5b8c1-d2d3-431e-8ed7-6e89c60a335d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:56.551818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5b8c1-d2d3-431e-8ed7-6e89c60a335d admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:56.552168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5b8c1-d2d3-431e-8ed7-6e89c60a335d admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:56.552456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5b8c1-d2d3-431e-8ed7-6e89c60a335d admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:56.553219 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5b8c1-d2d3-431e-8ed7-6e89c60a335d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:56.555366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5b8c1-d2d3-431e-8ed7-6e89c60a335d admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:56.580052 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1061/2105] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:37:56 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:37:56.584246 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0a5fc-1e52-437f-9b63-2a800ef0df74 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:56.595875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde0a5fc-1e52-437f-9b63-2a800ef0df74 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:56.600774 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379215fd-7853-4483-ad00-4a8175d32bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:56.601442 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379215fd-7853-4483-ad00-4a8175d32bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:56.601729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379215fd-7853-4483-ad00-4a8175d32bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:56.602010 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379215fd-7853-4483-ad00-4a8175d32bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:56.602733 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379215fd-7853-4483-ad00-4a8175d32bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:56.604463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379215fd-7853-4483-ad00-4a8175d32bd9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:56.616075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1045/2106] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:37:56.620290 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5b8c1-d2d3-431e-8ed7-6e89c60a335d admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:56.631405 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5b8c1-d2d3-431e-8ed7-6e89c60a335d admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:56.635950 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921445d2-1c66-4196-9106-09d13d38b350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:56.636916 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921445d2-1c66-4196-9106-09d13d38b350 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:56.637202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921445d2-1c66-4196-9106-09d13d38b350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:56.637481 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921445d2-1c66-4196-9106-09d13d38b350 admin admin] PATH_INFO: `/v3/projects/d17e57f09e7c42ca8dd90f1b9a79e79e/users/04274c6585644f36aeadf89b6c669bd4/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:56.651614 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921445d2-1c66-4196-9106-09d13d38b350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d17e57f09e7c42ca8dd90f1b9a79e79e, user_id=04274c6585644f36aeadf89b6c669bd4, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:56.654379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921445d2-1c66-4196-9106-09d13d38b350 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:56.663464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:56.669271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1062/2107] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:56 2022] PUT /identity/v3/projects/d17e57f09e7c42ca8dd90f1b9a79e79e/users/04274c6585644f36aeadf89b6c669bd4/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:56.673327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379215fd-7853-4483-ad00-4a8175d32bd9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:56.683453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379215fd-7853-4483-ad00-4a8175d32bd9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:56.688763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:56.740575 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98a92c-a570-4fe6-a8fd-2daa185b8b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:56.741235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98a92c-a570-4fe6-a8fd-2daa185b8b54 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:56.741522 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98a92c-a570-4fe6-a8fd-2daa185b8b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:56.741800 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98a92c-a570-4fe6-a8fd-2daa185b8b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:56.742513 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98a92c-a570-4fe6-a8fd-2daa185b8b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:56.744161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98a92c-a570-4fe6-a8fd-2daa185b8b54 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:56.755737 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1046/2108] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:37:56.759813 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921445d2-1c66-4196-9106-09d13d38b350 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:56.770113 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921445d2-1c66-4196-9106-09d13d38b350 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:56.774988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcace1-91e3-4487-90e6-c49a17b9ec72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:56.775844 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcace1-91e3-4487-90e6-c49a17b9ec72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:56.776135 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcace1-91e3-4487-90e6-c49a17b9ec72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:56.776422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dcace1-91e3-4487-90e6-c49a17b9ec72 admin admin] PATH_INFO: `/v3/projects/d17e57f09e7c42ca8dd90f1b9a79e79e/users/04274c6585644f36aeadf89b6c669bd4/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:56.779243 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcace1-91e3-4487-90e6-c49a17b9ec72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d17e57f09e7c42ca8dd90f1b9a79e79e, user_id=04274c6585644f36aeadf89b6c669bd4, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:56.781920 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dcace1-91e3-4487-90e6-c49a17b9ec72 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:56.792772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1063/2109] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:37:56 2022] PUT /identity/v3/projects/d17e57f09e7c42ca8dd90f1b9a79e79e/users/04274c6585644f36aeadf89b6c669bd4/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:56.804673 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f51b38-2398-43a0-8be7-ec5cfcb23826 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:56.805009 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f51b38-2398-43a0-8be7-ec5cfcb23826 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:56.805331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f51b38-2398-43a0-8be7-ec5cfcb23826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:56.832921 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-66f51b38-2398-43a0-8be7-ec5cfcb23826 None None] MFA Rules not processed for user `04274c6585644f36aeadf89b6c669bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:56.834233 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-66f51b38-2398-43a0-8be7-ec5cfcb23826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:56.907072 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1047/2110] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:56.919770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcace1-91e3-4487-90e6-c49a17b9ec72 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:56.921568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-28dcace1-91e3-4487-90e6-c49a17b9ec72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:56.988428 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dcace1-91e3-4487-90e6-c49a17b9ec72 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:56.992578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b632e5-eb74-41b3-a2a6-4d9b5ef6a49c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:56.993360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b632e5-eb74-41b3-a2a6-4d9b5ef6a49c service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:56.993652 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b632e5-eb74-41b3-a2a6-4d9b5ef6a49c service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:56.994639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b632e5-eb74-41b3-a2a6-4d9b5ef6a49c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:57.069618 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b632e5-eb74-41b3-a2a6-4d9b5ef6a49c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:57.071618 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b632e5-eb74-41b3-a2a6-4d9b5ef6a49c service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:57.082734 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1064/2111] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:58.633142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f51b38-2398-43a0-8be7-ec5cfcb23826 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:58.634654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-66f51b38-2398-43a0-8be7-ec5cfcb23826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:58.646843 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f51b38-2398-43a0-8be7-ec5cfcb23826 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:58.650422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df544a41-ac05-48ca-9e5e-c32bd852e4f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:58.651360 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df544a41-ac05-48ca-9e5e-c32bd852e4f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:58.651839 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df544a41-ac05-48ca-9e5e-c32bd852e4f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:58.652301 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df544a41-ac05-48ca-9e5e-c32bd852e4f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:58.712679 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df544a41-ac05-48ca-9e5e-c32bd852e4f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:58.714040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df544a41-ac05-48ca-9e5e-c32bd852e4f9 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:58.724302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1048/2112] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:59.079226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b632e5-eb74-41b3-a2a6-4d9b5ef6a49c service cinder] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:59.082127 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b0b632e5-eb74-41b3-a2a6-4d9b5ef6a49c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:59.136567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df544a41-ac05-48ca-9e5e-c32bd852e4f9 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:59.138538 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-df544a41-ac05-48ca-9e5e-c32bd852e4f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:59.169670 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b632e5-eb74-41b3-a2a6-4d9b5ef6a49c service cinder] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:59.174556 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd5294-a4ed-4aff-9578-a497d8e5ed24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:59.175547 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd5294-a4ed-4aff-9578-a497d8e5ed24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:59.175879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd5294-a4ed-4aff-9578-a497d8e5ed24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:59.176167 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd5294-a4ed-4aff-9578-a497d8e5ed24 admin admin] PATH_INFO: `/v3/users/05fa935ebbca444b9cc7457b605a3e3c` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:59.177512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd5294-a4ed-4aff-9578-a497d8e5ed24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05fa935ebbca444b9cc7457b605a3e3c)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:59.180068 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd5294-a4ed-4aff-9578-a497d8e5ed24 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:59.208658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:59.263519 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df544a41-ac05-48ca-9e5e-c32bd852e4f9 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:59.270060 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deed4e5-cb4d-4e57-8d43-f78338e1cec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:59.270801 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:59.270935 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deed4e5-cb4d-4e57-8d43-f78338e1cec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:59.271290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deed4e5-cb4d-4e57-8d43-f78338e1cec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:59.271632 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deed4e5-cb4d-4e57-8d43-f78338e1cec2 admin admin] PATH_INFO: `/v3/users/11d39989ad86401c8665f2a63f6920c0` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:59.273220 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deed4e5-cb4d-4e57-8d43-f78338e1cec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11d39989ad86401c8665f2a63f6920c0)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:59.276939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deed4e5-cb4d-4e57-8d43-f78338e1cec2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:59.277034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-9dfd5294-a4ed-4aff-9578-a497d8e5ed24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05fa935ebbca444b9cc7457b605a3e3c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:59.335506 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-9dfd5294-a4ed-4aff-9578-a497d8e5ed24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05fa935ebbca444b9cc7457b605a3e3c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:59.346138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-9dfd5294-a4ed-4aff-9578-a497d8e5ed24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05fa935ebbca444b9cc7457b605a3e3c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:59.353901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2deed4e5-cb4d-4e57-8d43-f78338e1cec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11d39989ad86401c8665f2a63f6920c0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:59.354070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-9dfd5294-a4ed-4aff-9578-a497d8e5ed24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05fa935ebbca444b9cc7457b605a3e3c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:59.356303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-9dfd5294-a4ed-4aff-9578-a497d8e5ed24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05fa935ebbca444b9cc7457b605a3e3c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:59.364253 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2deed4e5-cb4d-4e57-8d43-f78338e1cec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11d39989ad86401c8665f2a63f6920c0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:59.378014 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2deed4e5-cb4d-4e57-8d43-f78338e1cec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11d39989ad86401c8665f2a63f6920c0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:59.378841 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1065/2113] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:37:59 2022] DELETE /identity/v3/users/05fa935ebbca444b9cc7457b605a3e3c => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:59.379377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2deed4e5-cb4d-4e57-8d43-f78338e1cec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11d39989ad86401c8665f2a63f6920c0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:59.380385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2deed4e5-cb4d-4e57-8d43-f78338e1cec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11d39989ad86401c8665f2a63f6920c0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:59.383254 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd59d27-80c6-4de0-915f-5a6d2d065be6 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:59.383552 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd59d27-80c6-4de0-915f-5a6d2d065be6 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:59.383836 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd59d27-80c6-4de0-915f-5a6d2d065be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:59.405671 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-2fd59d27-80c6-4de0-915f-5a6d2d065be6 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:59.407126 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2fd59d27-80c6-4de0-915f-5a6d2d065be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:59.436588 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1049/2114] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:37:59 2022] DELETE /identity/v3/users/11d39989ad86401c8665f2a63f6920c0 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:59.440683 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deed4e5-cb4d-4e57-8d43-f78338e1cec2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:59.442290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2deed4e5-cb4d-4e57-8d43-f78338e1cec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:59.532988 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deed4e5-cb4d-4e57-8d43-f78338e1cec2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:59.536206 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1066/2115] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:37:59.537717 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be246862-740d-4a99-a020-8949e2edc1de None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:59.538112 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be246862-740d-4a99-a020-8949e2edc1de None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:59.538497 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be246862-740d-4a99-a020-8949e2edc1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:59.544174 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f3729-5dfe-4f5b-94cc-13db7dd575d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:59.546153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f3729-5dfe-4f5b-94cc-13db7dd575d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:59.548254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f3729-5dfe-4f5b-94cc-13db7dd575d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:59.548916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f3729-5dfe-4f5b-94cc-13db7dd575d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:59.552866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-753f3729-5dfe-4f5b-94cc-13db7dd575d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:59.564886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-be246862-740d-4a99-a020-8949e2edc1de None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:59.567511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-be246862-740d-4a99-a020-8949e2edc1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:59.652332 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f3729-5dfe-4f5b-94cc-13db7dd575d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:59.654763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f3729-5dfe-4f5b-94cc-13db7dd575d8 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:59.658361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1067/2116] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:59.659709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be246862-740d-4a99-a020-8949e2edc1de None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:59.659969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:59.663284 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-be246862-740d-4a99-a020-8949e2edc1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:59.668013 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1050/2117] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:37:59.670633 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f3729-5dfe-4f5b-94cc-13db7dd575d8 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:59.673105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-753f3729-5dfe-4f5b-94cc-13db7dd575d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:59.697925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f3729-5dfe-4f5b-94cc-13db7dd575d8 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:59.705006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3de54-ff74-4c43-9756-e78e4ef20419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:59.706231 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3de54-ff74-4c43-9756-e78e4ef20419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:59.706686 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3de54-ff74-4c43-9756-e78e4ef20419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:59.707103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3de54-ff74-4c43-9756-e78e4ef20419 admin admin] PATH_INFO: `/v3/users/7d8659a964d645ea99edec7bece0b7c1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:59.708431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d3de54-ff74-4c43-9756-e78e4ef20419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d8659a964d645ea99edec7bece0b7c1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:59.710797 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d3de54-ff74-4c43-9756-e78e4ef20419 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:59.736621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be246862-740d-4a99-a020-8949e2edc1de None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:59.740390 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55d671-f1cd-44e3-99fa-3e814c2e49e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:59.741038 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55d671-f1cd-44e3-99fa-3e814c2e49e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:59.741311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55d671-f1cd-44e3-99fa-3e814c2e49e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:59.741723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55d671-f1cd-44e3-99fa-3e814c2e49e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:59.754874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55d671-f1cd-44e3-99fa-3e814c2e49e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:59.756535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55d671-f1cd-44e3-99fa-3e814c2e49e3 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:59.768694 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1068/2118] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:59.772843 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-34d3de54-ff74-4c43-9756-e78e4ef20419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d8659a964d645ea99edec7bece0b7c1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:59.790644 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55d671-f1cd-44e3-99fa-3e814c2e49e3 service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:59.793183 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-34d3de54-ff74-4c43-9756-e78e4ef20419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d8659a964d645ea99edec7bece0b7c1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:59.798463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:37:59.804936 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55d671-f1cd-44e3-99fa-3e814c2e49e3 service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:37:59.807530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-34d3de54-ff74-4c43-9756-e78e4ef20419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d8659a964d645ea99edec7bece0b7c1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:59.811477 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-34d3de54-ff74-4c43-9756-e78e4ef20419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d8659a964d645ea99edec7bece0b7c1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:59.812571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-34d3de54-ff74-4c43-9756-e78e4ef20419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d8659a964d645ea99edec7bece0b7c1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:37:59.814661 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8383d7a-f259-434b-8be4-511008e73126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:37:59.815519 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8383d7a-f259-434b-8be4-511008e73126 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:59.816216 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8383d7a-f259-434b-8be4-511008e73126 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:59.816579 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8383d7a-f259-434b-8be4-511008e73126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:59.821251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e8383d7a-f259-434b-8be4-511008e73126 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:59.835638 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1051/2119] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:37:59 2022] DELETE /identity/v3/users/7d8659a964d645ea99edec7bece0b7c1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:37:59.839981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b047a-b7e5-415b-8764-9676596faaaf None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:37:59.840289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b047a-b7e5-415b-8764-9676596faaaf None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:37:59.840596 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b047a-b7e5-415b-8764-9676596faaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:37:59.842950 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8383d7a-f259-434b-8be4-511008e73126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:37:59.844690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8383d7a-f259-434b-8be4-511008e73126 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:37:59.859396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c75b047a-b7e5-415b-8764-9676596faaaf None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:37:59.860795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c75b047a-b7e5-415b-8764-9676596faaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:59.861078 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1069/2120] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:37:59.961462 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1052/2121] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:37:59.973868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8383d7a-f259-434b-8be4-511008e73126 service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:37:59.976855 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e8383d7a-f259-434b-8be4-511008e73126 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:37:59.990412 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8383d7a-f259-434b-8be4-511008e73126 service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:00.067691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2a27b-4a67-4f20-b207-faacb617500a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:00.068361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2a27b-4a67-4f20-b207-faacb617500a service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:00.068715 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2a27b-4a67-4f20-b207-faacb617500a service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:00.069522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2a27b-4a67-4f20-b207-faacb617500a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:00.082218 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2a27b-4a67-4f20-b207-faacb617500a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:00.083770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2a27b-4a67-4f20-b207-faacb617500a service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:00.097072 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1070/2122] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:37:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:00.265042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b047a-b7e5-415b-8764-9676596faaaf None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:00.266498 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c75b047a-b7e5-415b-8764-9676596faaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:00.306339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b047a-b7e5-415b-8764-9676596faaaf None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:00.316946 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab7ffc-5d7c-4f2a-94d7-46f352e30095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:00.319404 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab7ffc-5d7c-4f2a-94d7-46f352e30095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:00.319696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab7ffc-5d7c-4f2a-94d7-46f352e30095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:00.319978 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab7ffc-5d7c-4f2a-94d7-46f352e30095 admin admin] PATH_INFO: `/v3/projects/3163d606f2a44729bc4233611cc83e03` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:00.321186 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab7ffc-5d7c-4f2a-94d7-46f352e30095 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3163d606f2a44729bc4233611cc83e03)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:00.323396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab7ffc-5d7c-4f2a-94d7-46f352e30095 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:00.396712 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2a27b-4a67-4f20-b207-faacb617500a service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:00.401262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-daa2a27b-4a67-4f20-b207-faacb617500a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:00.401664 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1bab7ffc-5d7c-4f2a-94d7-46f352e30095 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3163d606f2a44729bc4233611cc83e03', 'request_id': 'req-1bab7ffc-5d7c-4f2a-94d7-46f352e30095'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:00.413875 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1bab7ffc-5d7c-4f2a-94d7-46f352e30095 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3163d606f2a44729bc4233611cc83e03', 'request_id': 'req-1bab7ffc-5d7c-4f2a-94d7-46f352e30095'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:00.425636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1bab7ffc-5d7c-4f2a-94d7-46f352e30095 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3163d606f2a44729bc4233611cc83e03', 'request_id': 'req-1bab7ffc-5d7c-4f2a-94d7-46f352e30095'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:00.428275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:00.446419 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2a27b-4a67-4f20-b207-faacb617500a service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:00.464833 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1bab7ffc-5d7c-4f2a-94d7-46f352e30095 admin admin] The token cache is being invalidate because project 3163d606f2a44729bc4233611cc83e03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:00.466496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1bab7ffc-5d7c-4f2a-94d7-46f352e30095 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:00.468792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1bab7ffc-5d7c-4f2a-94d7-46f352e30095 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:00.472206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1053/2123] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:00 2022] DELETE /identity/v3/projects/3163d606f2a44729bc4233611cc83e03 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:00.477230 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebcc31-fea0-4e69-a723-d0b85876d8a1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:00.477847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebcc31-fea0-4e69-a723-d0b85876d8a1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:00.478267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebcc31-fea0-4e69-a723-d0b85876d8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:00.523062 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b2ebcc31-fea0-4e69-a723-d0b85876d8a1 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:00.525540 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b2ebcc31-fea0-4e69-a723-d0b85876d8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:00.614672 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab756e2-b5ed-4a19-8182-92d0ef565c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:00.617731 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab756e2-b5ed-4a19-8182-92d0ef565c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:00.617731 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab756e2-b5ed-4a19-8182-92d0ef565c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:00.617731 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab756e2-b5ed-4a19-8182-92d0ef565c4f admin admin] PATH_INFO: `/v3/projects/3dd4c9c12f6845f0b8e0e030a2d87e9d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:00.618554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab756e2-b5ed-4a19-8182-92d0ef565c4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dd4c9c12f6845f0b8e0e030a2d87e9d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:00.620944 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab756e2-b5ed-4a19-8182-92d0ef565c4f admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:00.654850 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1054/2124] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:38:00.663218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8caf9-8159-46b5-9c25-ebaded3b7353 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:00.663521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8caf9-8159-46b5-9c25-ebaded3b7353 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:00.663812 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8caf9-8159-46b5-9c25-ebaded3b7353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:00.684757 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6fa8caf9-8159-46b5-9c25-ebaded3b7353 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:00.684884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-6ab756e2-b5ed-4a19-8182-92d0ef565c4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dd4c9c12f6845f0b8e0e030a2d87e9d', 'request_id': 'req-6ab756e2-b5ed-4a19-8182-92d0ef565c4f'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:00.685050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6fa8caf9-8159-46b5-9c25-ebaded3b7353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:00.693335 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-6ab756e2-b5ed-4a19-8182-92d0ef565c4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dd4c9c12f6845f0b8e0e030a2d87e9d', 'request_id': 'req-6ab756e2-b5ed-4a19-8182-92d0ef565c4f'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:00.704763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-6ab756e2-b5ed-4a19-8182-92d0ef565c4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dd4c9c12f6845f0b8e0e030a2d87e9d', 'request_id': 'req-6ab756e2-b5ed-4a19-8182-92d0ef565c4f'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:00.720809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-6ab756e2-b5ed-4a19-8182-92d0ef565c4f admin admin] The token cache is being invalidate because project 3dd4c9c12f6845f0b8e0e030a2d87e9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:00.721522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-6ab756e2-b5ed-4a19-8182-92d0ef565c4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:00.723892 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-6ab756e2-b5ed-4a19-8182-92d0ef565c4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:00.726975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1071/2125] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:00 2022] DELETE /identity/v3/projects/3dd4c9c12f6845f0b8e0e030a2d87e9d => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:00.732126 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6465295-11c5-4430-b638-c9104e76a946 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:00.734025 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6465295-11c5-4430-b638-c9104e76a946 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:00.735600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6465295-11c5-4430-b638-c9104e76a946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:00.765729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-d6465295-11c5-4430-b638-c9104e76a946 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:00.767003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d6465295-11c5-4430-b638-c9104e76a946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:00.788414 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1055/2126] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:00.794178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8caf9-8159-46b5-9c25-ebaded3b7353 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:00.796179 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:00.798545 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6fa8caf9-8159-46b5-9c25-ebaded3b7353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:00.814625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8caf9-8159-46b5-9c25-ebaded3b7353 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:00.820168 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2777f4-d911-4787-a98a-ddac984403a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:00.821958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2777f4-d911-4787-a98a-ddac984403a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:00.823143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2777f4-d911-4787-a98a-ddac984403a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:00.823686 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2777f4-d911-4787-a98a-ddac984403a4 admin admin] PATH_INFO: `/v3/projects/5b37e6edb6044254afd6341775b46e17` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:00.829197 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2777f4-d911-4787-a98a-ddac984403a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b37e6edb6044254afd6341775b46e17)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:00.831497 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2777f4-d911-4787-a98a-ddac984403a4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:00.877675 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1072/2127] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:38:00.878828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6465295-11c5-4430-b638-c9104e76a946 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:00.891252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6465295-11c5-4430-b638-c9104e76a946 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:00.897529 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4f2777f4-d911-4787-a98a-ddac984403a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b37e6edb6044254afd6341775b46e17', 'request_id': 'req-4f2777f4-d911-4787-a98a-ddac984403a4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:00.907225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4f2777f4-d911-4787-a98a-ddac984403a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b37e6edb6044254afd6341775b46e17', 'request_id': 'req-4f2777f4-d911-4787-a98a-ddac984403a4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:00.937329 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4f2777f4-d911-4787-a98a-ddac984403a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b37e6edb6044254afd6341775b46e17', 'request_id': 'req-4f2777f4-d911-4787-a98a-ddac984403a4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:00.971452 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4f2777f4-d911-4787-a98a-ddac984403a4 admin admin] The token cache is being invalidate because project 5b37e6edb6044254afd6341775b46e17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:00.974042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4f2777f4-d911-4787-a98a-ddac984403a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:00.975638 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4f2777f4-d911-4787-a98a-ddac984403a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:00.978402 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1056/2128] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:00 2022] DELETE /identity/v3/projects/5b37e6edb6044254afd6341775b46e17 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:00.981647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1684ea5-098b-411f-9b06-8b8c0bff7e87 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:00.981822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1684ea5-098b-411f-9b06-8b8c0bff7e87 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:00.984339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1684ea5-098b-411f-9b06-8b8c0bff7e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:01.013252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abee57e-bf96-44eb-bb47-6d7d6bf62e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:01.014993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abee57e-bf96-44eb-bb47-6d7d6bf62e9c admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:01.016069 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abee57e-bf96-44eb-bb47-6d7d6bf62e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:01.016575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abee57e-bf96-44eb-bb47-6d7d6bf62e9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:01.017154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c1684ea5-098b-411f-9b06-8b8c0bff7e87 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:01.018151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abee57e-bf96-44eb-bb47-6d7d6bf62e9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:38:01.018528 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c1684ea5-098b-411f-9b06-8b8c0bff7e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:01.019134 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abee57e-bf96-44eb-bb47-6d7d6bf62e9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:01.021583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abee57e-bf96-44eb-bb47-6d7d6bf62e9c admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:01.033844 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1057/2129] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:01.035123 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1684ea5-098b-411f-9b06-8b8c0bff7e87 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:01.036679 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c1684ea5-098b-411f-9b06-8b8c0bff7e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:01.045303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1073/2130] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:38:01.046501 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abee57e-bf96-44eb-bb47-6d7d6bf62e9c admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:01.055589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1684ea5-098b-411f-9b06-8b8c0bff7e87 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:01.059704 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abee57e-bf96-44eb-bb47-6d7d6bf62e9c admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:01.061753 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651847c-d261-477a-8aca-bd95c5405bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:01.062703 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651847c-d261-477a-8aca-bd95c5405bb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:01.063031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651847c-d261-477a-8aca-bd95c5405bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:01.063313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651847c-d261-477a-8aca-bd95c5405bb0 admin admin] PATH_INFO: `/v3/users/236da07c2b7043419aed2d9e83493502` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:01.064856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651847c-d261-477a-8aca-bd95c5405bb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=236da07c2b7043419aed2d9e83493502)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:01.066883 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651847c-d261-477a-8aca-bd95c5405bb0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:01.073790 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51eab8-6335-4f5d-ad4e-0362500a0e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:01.073950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:01.074797 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51eab8-6335-4f5d-ad4e-0362500a0e36 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:01.075134 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51eab8-6335-4f5d-ad4e-0362500a0e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:01.075428 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51eab8-6335-4f5d-ad4e-0362500a0e36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:01.076171 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51eab8-6335-4f5d-ad4e-0362500a0e36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:38:01.076485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51eab8-6335-4f5d-ad4e-0362500a0e36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:01.078167 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51eab8-6335-4f5d-ad4e-0362500a0e36 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:01.111849 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1074/2131] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:38:01.113020 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51eab8-6335-4f5d-ad4e-0362500a0e36 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:01.115268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8d51eab8-6335-4f5d-ad4e-0362500a0e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:01.132467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51eab8-6335-4f5d-ad4e-0362500a0e36 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:01.142197 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b3298-9b86-4bfd-a377-0102e06dc7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:01.142873 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b3298-9b86-4bfd-a377-0102e06dc7d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:01.143197 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b3298-9b86-4bfd-a377-0102e06dc7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:01.143870 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b3298-9b86-4bfd-a377-0102e06dc7d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:01.144674 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b3298-9b86-4bfd-a377-0102e06dc7d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:01.147150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b3298-9b86-4bfd-a377-0102e06dc7d9 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:01.154694 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f651847c-d261-477a-8aca-bd95c5405bb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '236da07c2b7043419aed2d9e83493502'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:01.168605 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f651847c-d261-477a-8aca-bd95c5405bb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '236da07c2b7043419aed2d9e83493502'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:01.185418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f651847c-d261-477a-8aca-bd95c5405bb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '236da07c2b7043419aed2d9e83493502'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:01.186222 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f651847c-d261-477a-8aca-bd95c5405bb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '236da07c2b7043419aed2d9e83493502'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:01.187006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f651847c-d261-477a-8aca-bd95c5405bb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '236da07c2b7043419aed2d9e83493502'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:01.216980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1058/2132] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:38:01 2022] DELETE /identity/v3/users/236da07c2b7043419aed2d9e83493502 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:01.218336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1075/2133] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:38:01 2022] POST /identity/v3/projects => generated 399 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:01.221551 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651847c-d261-477a-8aca-bd95c5405bb0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:01.222643 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b3298-9b86-4bfd-a377-0102e06dc7d9 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:01.223864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f651847c-d261-477a-8aca-bd95c5405bb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:01.225786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-9b1b3298-9b86-4bfd-a377-0102e06dc7d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:01.319198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b3298-9b86-4bfd-a377-0102e06dc7d9 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:01.319412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651847c-d261-477a-8aca-bd95c5405bb0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:01.323178 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67a6b4-048a-44b6-8b13-9e49dc19125c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:01.323950 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67a6b4-048a-44b6-8b13-9e49dc19125c admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:01.325210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67a6b4-048a-44b6-8b13-9e49dc19125c admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:01.325210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67a6b4-048a-44b6-8b13-9e49dc19125c admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:01.325875 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67a6b4-048a-44b6-8b13-9e49dc19125c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:01.326796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74629e-173d-42b5-a30e-8c5d46b28c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:01.327652 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74629e-173d-42b5-a30e-8c5d46b28c4a admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:01.327836 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67a6b4-048a-44b6-8b13-9e49dc19125c admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:01.328131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74629e-173d-42b5-a30e-8c5d46b28c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:01.328514 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74629e-173d-42b5-a30e-8c5d46b28c4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:01.329365 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74629e-173d-42b5-a30e-8c5d46b28c4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:01.331437 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74629e-173d-42b5-a30e-8c5d46b28c4a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:01.350744 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1076/2134] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:38:01 2022] POST /identity/v3/users => generated 300 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:38:01.355402 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67a6b4-048a-44b6-8b13-9e49dc19125c admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:01.367188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1059/2135] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:38:01 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:01.368491 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74629e-173d-42b5-a30e-8c5d46b28c4a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:01.370037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67a6b4-048a-44b6-8b13-9e49dc19125c admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:01.383093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:01.480751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74629e-173d-42b5-a30e-8c5d46b28c4a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:01.488650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c6ffc-f0a7-4bbe-a7c2-46d6fb176dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:01.489586 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c6ffc-f0a7-4bbe-a7c2-46d6fb176dac admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:01.489997 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c6ffc-f0a7-4bbe-a7c2-46d6fb176dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:01.490527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c6ffc-f0a7-4bbe-a7c2-46d6fb176dac admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:01.491487 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c6ffc-f0a7-4bbe-a7c2-46d6fb176dac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:01.494149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c6ffc-f0a7-4bbe-a7c2-46d6fb176dac admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:01.501850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5254c-29e4-4c9e-a8f9-363aacf84b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:01.503579 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5254c-29e4-4c9e-a8f9-363aacf84b0b admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:01.504122 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5254c-29e4-4c9e-a8f9-363aacf84b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:01.505024 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5254c-29e4-4c9e-a8f9-363aacf84b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:01.509554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5254c-29e4-4c9e-a8f9-363aacf84b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:01.512780 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5254c-29e4-4c9e-a8f9-363aacf84b0b admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:01.517151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:01.528516 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1077/2136] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:01.529878 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1060/2137] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:38:01 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:01.532449 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5254c-29e4-4c9e-a8f9-363aacf84b0b admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:01.536228 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c6ffc-f0a7-4bbe-a7c2-46d6fb176dac admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:01.546894 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5254c-29e4-4c9e-a8f9-363aacf84b0b admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:01.552054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c6ffc-f0a7-4bbe-a7c2-46d6fb176dac admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:01.552973 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ec340-5c5a-479b-b669-95f329025bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:01.553730 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ec340-5c5a-479b-b669-95f329025bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:01.554098 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ec340-5c5a-479b-b669-95f329025bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:01.554448 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ec340-5c5a-479b-b669-95f329025bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:01.555298 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ec340-5c5a-479b-b669-95f329025bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:01.556796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ec671-ec61-4274-8310-193e515de2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:01.557268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ec340-5c5a-479b-b669-95f329025bc1 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:01.558008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ec671-ec61-4274-8310-193e515de2dd admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:01.558671 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ec671-ec61-4274-8310-193e515de2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:01.559172 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ec671-ec61-4274-8310-193e515de2dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:01.560404 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ec671-ec61-4274-8310-193e515de2dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:01.562134 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ec671-ec61-4274-8310-193e515de2dd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:01.570017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1078/2138] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:01.573941 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1061/2139] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:38:01.574163 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ec340-5c5a-479b-b669-95f329025bc1 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:01.577743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ec671-ec61-4274-8310-193e515de2dd admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:01.584803 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ec340-5c5a-479b-b669-95f329025bc1 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:01.588823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ec671-ec61-4274-8310-193e515de2dd admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:01.589602 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ba372-774e-4ecc-9e9f-0c29836d9c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:01.591443 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ba372-774e-4ecc-9e9f-0c29836d9c9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:01.591796 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ba372-774e-4ecc-9e9f-0c29836d9c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:01.592088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ba372-774e-4ecc-9e9f-0c29836d9c9b admin admin] PATH_INFO: `/v3/projects/052e490afbed45a783fd7adc1e08823f/users/b15d667a30ee40beb7aa2affcb490fc8/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:01.595145 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620733ed-dd2c-4d7d-8555-bee25706b31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:01.595890 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620733ed-dd2c-4d7d-8555-bee25706b31a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:01.596280 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620733ed-dd2c-4d7d-8555-bee25706b31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:01.596567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620733ed-dd2c-4d7d-8555-bee25706b31a admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:01.597316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620733ed-dd2c-4d7d-8555-bee25706b31a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:01.598956 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620733ed-dd2c-4d7d-8555-bee25706b31a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:01.609081 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ba372-774e-4ecc-9e9f-0c29836d9c9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=052e490afbed45a783fd7adc1e08823f, user_id=b15d667a30ee40beb7aa2affcb490fc8, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:01.612069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1062/2140] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:01.612209 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ba372-774e-4ecc-9e9f-0c29836d9c9b admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:01.616224 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620733ed-dd2c-4d7d-8555-bee25706b31a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:01.628259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620733ed-dd2c-4d7d-8555-bee25706b31a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:01.630772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1079/2141] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:01 2022] PUT /identity/v3/projects/052e490afbed45a783fd7adc1e08823f/users/b15d667a30ee40beb7aa2affcb490fc8/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:38:01.643405 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74d2e7-9773-4750-af8d-4e9d5fd9bef8 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:01.643600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74d2e7-9773-4750-af8d-4e9d5fd9bef8 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:01.644056 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74d2e7-9773-4750-af8d-4e9d5fd9bef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:01.678992 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-fe74d2e7-9773-4750-af8d-4e9d5fd9bef8 None None] MFA Rules not processed for user `b15d667a30ee40beb7aa2affcb490fc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:01.680297 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-fe74d2e7-9773-4750-af8d-4e9d5fd9bef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:01.703473 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93dd40b-1132-4d81-b3d9-a171bbebe664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:01.704477 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dd40b-1132-4d81-b3d9-a171bbebe664 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:01.704985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dd40b-1132-4d81-b3d9-a171bbebe664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:01.705276 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dd40b-1132-4d81-b3d9-a171bbebe664 admin admin] PATH_INFO: `/v3/projects/f7bc6b504b764fe6b3f892e2a8a88a05/users/34f24b9b6f2d45c4907ce8cb419ab4a4/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:01.726140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93dd40b-1132-4d81-b3d9-a171bbebe664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7bc6b504b764fe6b3f892e2a8a88a05, user_id=34f24b9b6f2d45c4907ce8cb419ab4a4, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:01.729976 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93dd40b-1132-4d81-b3d9-a171bbebe664 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:01.749076 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1063/2142] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:01 2022] PUT /identity/v3/projects/f7bc6b504b764fe6b3f892e2a8a88a05/users/34f24b9b6f2d45c4907ce8cb419ab4a4/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:38:01.750050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93dd40b-1132-4d81-b3d9-a171bbebe664 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:01.752252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f93dd40b-1132-4d81-b3d9-a171bbebe664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:01.776063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1080/2143] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:38:01.777684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554313c-5f0f-4c16-ae4c-643a3fb92ddd None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:01.778160 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554313c-5f0f-4c16-ae4c-643a3fb92ddd None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:01.778464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554313c-5f0f-4c16-ae4c-643a3fb92ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:01.819335 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-e554313c-5f0f-4c16-ae4c-643a3fb92ddd None None] MFA Rules not processed for user `34f24b9b6f2d45c4907ce8cb419ab4a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:01.821280 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e554313c-5f0f-4c16-ae4c-643a3fb92ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:01.874260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93dd40b-1132-4d81-b3d9-a171bbebe664 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:01.879199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b97d1b-b41d-4f63-ba34-6776636bdcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:01.880135 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b97d1b-b41d-4f63-ba34-6776636bdcf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:01.880667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b97d1b-b41d-4f63-ba34-6776636bdcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:01.880856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b97d1b-b41d-4f63-ba34-6776636bdcf4 admin admin] PATH_INFO: `/v3/projects/977203a35a3d4fb0b94566759835e600` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:01.882299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b97d1b-b41d-4f63-ba34-6776636bdcf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=977203a35a3d4fb0b94566759835e600)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:01.884458 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b97d1b-b41d-4f63-ba34-6776636bdcf4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:01.899402 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:01.912868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1081/2144] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:01.915030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554313c-5f0f-4c16-ae4c-643a3fb92ddd None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:01.917444 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e554313c-5f0f-4c16-ae4c-643a3fb92ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:01.949211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a2b97d1b-b41d-4f63-ba34-6776636bdcf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '977203a35a3d4fb0b94566759835e600', 'request_id': 'req-a2b97d1b-b41d-4f63-ba34-6776636bdcf4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:01.957627 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a2b97d1b-b41d-4f63-ba34-6776636bdcf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '977203a35a3d4fb0b94566759835e600', 'request_id': 'req-a2b97d1b-b41d-4f63-ba34-6776636bdcf4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:01.962398 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:01.965690 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a2b97d1b-b41d-4f63-ba34-6776636bdcf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '977203a35a3d4fb0b94566759835e600', 'request_id': 'req-a2b97d1b-b41d-4f63-ba34-6776636bdcf4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:01.984591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a2b97d1b-b41d-4f63-ba34-6776636bdcf4 admin admin] The token cache is being invalidate because project 977203a35a3d4fb0b94566759835e600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:01.984908 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a2b97d1b-b41d-4f63-ba34-6776636bdcf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:01.988182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a2b97d1b-b41d-4f63-ba34-6776636bdcf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:01.988678 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1064/2145] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:01 2022] DELETE /identity/v3/projects/977203a35a3d4fb0b94566759835e600 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:01.993418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5db0a-acd1-4072-9281-33914808f6a3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:01.993716 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5db0a-acd1-4072-9281-33914808f6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:01.994005 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5db0a-acd1-4072-9281-33914808f6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:01.994154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554313c-5f0f-4c16-ae4c-643a3fb92ddd None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:02.013202 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1eb5db0a-acd1-4072-9281-33914808f6a3 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:02.014456 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1eb5db0a-acd1-4072-9281-33914808f6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:02.063066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab31ee-6403-4b04-8e5a-34a5215b6c54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:02.063773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab31ee-6403-4b04-8e5a-34a5215b6c54 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.064205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab31ee-6403-4b04-8e5a-34a5215b6c54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:02.064517 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab31ee-6403-4b04-8e5a-34a5215b6c54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:02.066561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-6bab31ee-6403-4b04-8e5a-34a5215b6c54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:02.091404 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1065/2146] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:38:02.100265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3147275-81cc-47bb-9514-2554764803d0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.100743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3147275-81cc-47bb-9514-2554764803d0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:02.101275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3147275-81cc-47bb-9514-2554764803d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:02.129486 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d3147275-81cc-47bb-9514-2554764803d0 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:02.130712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d3147275-81cc-47bb-9514-2554764803d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:02.142970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1066/2147] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:02.142970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3147275-81cc-47bb-9514-2554764803d0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:02.144879 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d3147275-81cc-47bb-9514-2554764803d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:02.146070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab31ee-6403-4b04-8e5a-34a5215b6c54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:02.147629 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab31ee-6403-4b04-8e5a-34a5215b6c54 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:02.157754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1082/2148] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:01 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:02.158575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab31ee-6403-4b04-8e5a-34a5215b6c54 service cinder] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:02.160003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3147275-81cc-47bb-9514-2554764803d0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:02.164289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c5f6c-2576-4097-b7ab-443eccd750a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:02.165039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c5f6c-2576-4097-b7ab-443eccd750a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.165752 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c5f6c-2576-4097-b7ab-443eccd750a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:02.165752 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c5f6c-2576-4097-b7ab-443eccd750a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:02.167350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e40c5f6c-2576-4097-b7ab-443eccd750a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:02.170389 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab31ee-6403-4b04-8e5a-34a5215b6c54 service cinder] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:02.175393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c869662-422a-4813-a9cc-231861324074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:02.176060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c869662-422a-4813-a9cc-231861324074 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.176346 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c869662-422a-4813-a9cc-231861324074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:02.176627 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c869662-422a-4813-a9cc-231861324074 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:02.177351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c869662-422a-4813-a9cc-231861324074 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:38:02.177664 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c869662-422a-4813-a9cc-231861324074 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:02.179280 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c869662-422a-4813-a9cc-231861324074 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:02.192017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1083/2149] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:38:02.196311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c869662-422a-4813-a9cc-231861324074 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:02.206756 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c869662-422a-4813-a9cc-231861324074 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:02.211450 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c3d11-0427-433c-a322-2e0f23ef89e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:02.212112 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c3d11-0427-433c-a322-2e0f23ef89e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.212400 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c3d11-0427-433c-a322-2e0f23ef89e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:02.212683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c3d11-0427-433c-a322-2e0f23ef89e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:02.213381 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c3d11-0427-433c-a322-2e0f23ef89e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:02.216550 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38c3d11-0427-433c-a322-2e0f23ef89e5 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:02.234758 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c5f6c-2576-4097-b7ab-443eccd750a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:02.236247 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c5f6c-2576-4097-b7ab-443eccd750a5 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:02.245760 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1067/2150] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:02.250742 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1084/2151] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:38:02 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:38:02.259902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c5f6c-2576-4097-b7ab-443eccd750a5 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:02.270999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c5f6c-2576-4097-b7ab-443eccd750a5 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:02.343636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2274dde-30ec-46e5-aff3-9fe318c2d9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:02.344653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2274dde-30ec-46e5-aff3-9fe318c2d9f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.344941 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2274dde-30ec-46e5-aff3-9fe318c2d9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:02.345224 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2274dde-30ec-46e5-aff3-9fe318c2d9f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:02.346099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2274dde-30ec-46e5-aff3-9fe318c2d9f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:02.349353 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2274dde-30ec-46e5-aff3-9fe318c2d9f2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:02.380250 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1068/2152] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:38:02 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:02.381504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c3d11-0427-433c-a322-2e0f23ef89e5 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:02.392666 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c3d11-0427-433c-a322-2e0f23ef89e5 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:02.397574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e9c76-40d6-49b7-b878-c6e5a580faf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:02.398249 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e9c76-40d6-49b7-b878-c6e5a580faf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.398530 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e9c76-40d6-49b7-b878-c6e5a580faf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:02.398809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e9c76-40d6-49b7-b878-c6e5a580faf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:02.399648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e9c76-40d6-49b7-b878-c6e5a580faf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:02.401332 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e9c76-40d6-49b7-b878-c6e5a580faf3 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:02.413294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1085/2153] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:02.417546 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2274dde-30ec-46e5-aff3-9fe318c2d9f2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:02.428913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2274dde-30ec-46e5-aff3-9fe318c2d9f2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:02.434571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb8ec0-9086-4340-82cf-626e32a29276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:02.435483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb8ec0-9086-4340-82cf-626e32a29276 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.435778 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb8ec0-9086-4340-82cf-626e32a29276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:02.436225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb8ec0-9086-4340-82cf-626e32a29276 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:02.436993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb8ec0-9086-4340-82cf-626e32a29276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:02.438822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb8ec0-9086-4340-82cf-626e32a29276 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:02.446015 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:02.458407 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1069/2154] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:02.459466 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e9c76-40d6-49b7-b878-c6e5a580faf3 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:02.471356 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e9c76-40d6-49b7-b878-c6e5a580faf3 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:02.488996 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19846a69-e6ce-4fe0-b224-7f89c07620e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:02.490176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19846a69-e6ce-4fe0-b224-7f89c07620e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.490709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19846a69-e6ce-4fe0-b224-7f89c07620e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:02.490864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19846a69-e6ce-4fe0-b224-7f89c07620e4 admin admin] PATH_INFO: `/v3/projects/527d0b7e41ec40b99c9b2f383a0baa73/users/25569f5a20244d9380aaed41b7278e19/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:02.523457 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19846a69-e6ce-4fe0-b224-7f89c07620e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=527d0b7e41ec40b99c9b2f383a0baa73, user_id=25569f5a20244d9380aaed41b7278e19, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:02.526600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19846a69-e6ce-4fe0-b224-7f89c07620e4 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:02.529784 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:02.543559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1086/2155] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:02 2022] PUT /identity/v3/projects/527d0b7e41ec40b99c9b2f383a0baa73/users/25569f5a20244d9380aaed41b7278e19/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:02.562427 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ba83f-fea2-4692-8b56-cfb7e22cf1a6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.562427 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ba83f-fea2-4692-8b56-cfb7e22cf1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:02.562427 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ba83f-fea2-4692-8b56-cfb7e22cf1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:02.603385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ba3ba83f-fea2-4692-8b56-cfb7e22cf1a6 None None] MFA Rules not processed for user `25569f5a20244d9380aaed41b7278e19`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:02.604247 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ba3ba83f-fea2-4692-8b56-cfb7e22cf1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:02.691856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1070/2156] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:02.700492 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19846a69-e6ce-4fe0-b224-7f89c07620e4 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:02.781986 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19846a69-e6ce-4fe0-b224-7f89c07620e4 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:02.792203 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfa724-d9f2-40fa-8031-89d98bca1563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:02.793181 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfa724-d9f2-40fa-8031-89d98bca1563 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.794226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfa724-d9f2-40fa-8031-89d98bca1563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:02.794662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfa724-d9f2-40fa-8031-89d98bca1563 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:02.795693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dfa724-d9f2-40fa-8031-89d98bca1563 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:02.798727 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dfa724-d9f2-40fa-8031-89d98bca1563 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:02.834654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1087/2157] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:38:02 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:02.836433 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ba83f-fea2-4692-8b56-cfb7e22cf1a6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:02.850823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ba83f-fea2-4692-8b56-cfb7e22cf1a6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:02.959402 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5aba6-2d56-4e6b-82ab-73215402ca3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:02.960089 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5aba6-2d56-4e6b-82ab-73215402ca3e admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:02.960385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5aba6-2d56-4e6b-82ab-73215402ca3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:02.960670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5aba6-2d56-4e6b-82ab-73215402ca3e admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:02.961368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5aba6-2d56-4e6b-82ab-73215402ca3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:02.963307 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5aba6-2d56-4e6b-82ab-73215402ca3e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:02.986968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1071/2158] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:38:02 2022] POST /identity/v3/users => generated 294 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:02.990580 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfa724-d9f2-40fa-8031-89d98bca1563 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:03.002394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfa724-d9f2-40fa-8031-89d98bca1563 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:03.007015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b35b3d-25d0-483a-baf1-07628c4dcb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:03.007718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b35b3d-25d0-483a-baf1-07628c4dcb70 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:03.008005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b35b3d-25d0-483a-baf1-07628c4dcb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:03.008400 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b35b3d-25d0-483a-baf1-07628c4dcb70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:03.009131 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b35b3d-25d0-483a-baf1-07628c4dcb70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:03.010730 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b35b3d-25d0-483a-baf1-07628c4dcb70 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:03.023429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1088/2159] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:03.027635 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5aba6-2d56-4e6b-82ab-73215402ca3e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:03.039014 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5aba6-2d56-4e6b-82ab-73215402ca3e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:03.044056 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d725bf1-3210-44d2-889b-1ad2d56c959c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:03.044874 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d725bf1-3210-44d2-889b-1ad2d56c959c admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:03.045057 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d725bf1-3210-44d2-889b-1ad2d56c959c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:03.045334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d725bf1-3210-44d2-889b-1ad2d56c959c admin admin] PATH_INFO: `/v3/projects/0e158e53194f4953a47067a045c3aa9b/users/24704fe11803499e852ecdd888977856/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:03.063990 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d725bf1-3210-44d2-889b-1ad2d56c959c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e158e53194f4953a47067a045c3aa9b, user_id=24704fe11803499e852ecdd888977856, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:03.067010 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d725bf1-3210-44d2-889b-1ad2d56c959c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:03.080655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1072/2160] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:03 2022] PUT /identity/v3/projects/0e158e53194f4953a47067a045c3aa9b/users/24704fe11803499e852ecdd888977856/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:38:03.084539 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b35b3d-25d0-483a-baf1-07628c4dcb70 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:03.096043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b35b3d-25d0-483a-baf1-07628c4dcb70 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:03.171844 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4aaf12-a8c6-487c-97b5-60832496fca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:03.173929 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4aaf12-a8c6-487c-97b5-60832496fca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:03.174436 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4aaf12-a8c6-487c-97b5-60832496fca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:03.175042 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4aaf12-a8c6-487c-97b5-60832496fca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:03.176478 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4aaf12-a8c6-487c-97b5-60832496fca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:03.178356 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4aaf12-a8c6-487c-97b5-60832496fca9 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:03.194568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1089/2161] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:03.196211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d725bf1-3210-44d2-889b-1ad2d56c959c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:03.213467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d725bf1-3210-44d2-889b-1ad2d56c959c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:03.219176 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d0d6e-9a67-48d0-b8cb-9c61d04995ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:03.220165 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d0d6e-9a67-48d0-b8cb-9c61d04995ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:03.220525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d0d6e-9a67-48d0-b8cb-9c61d04995ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:03.220882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d0d6e-9a67-48d0-b8cb-9c61d04995ca admin admin] PATH_INFO: `/v3/projects/0e158e53194f4953a47067a045c3aa9b/users/24704fe11803499e852ecdd888977856/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:03.223031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d0d6e-9a67-48d0-b8cb-9c61d04995ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e158e53194f4953a47067a045c3aa9b, user_id=24704fe11803499e852ecdd888977856, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:03.225749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d0d6e-9a67-48d0-b8cb-9c61d04995ca admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:03.237648 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1073/2162] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:03 2022] PUT /identity/v3/projects/0e158e53194f4953a47067a045c3aa9b/users/24704fe11803499e852ecdd888977856/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:03.249292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dd2ed-b77c-4379-a852-59a6ebec94d6 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:03.249678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dd2ed-b77c-4379-a852-59a6ebec94d6 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:03.250070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52dd2ed-b77c-4379-a852-59a6ebec94d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:03.294288 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-f52dd2ed-b77c-4379-a852-59a6ebec94d6 None None] MFA Rules not processed for user `24704fe11803499e852ecdd888977856`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:03.297359 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-f52dd2ed-b77c-4379-a852-59a6ebec94d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:03.361669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:03.382757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1090/2163] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:03.393471 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d0d6e-9a67-48d0-b8cb-9c61d04995ca admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:03.397617 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4e9d0d6e-9a67-48d0-b8cb-9c61d04995ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:03.449022 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:03.462340 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d0d6e-9a67-48d0-b8cb-9c61d04995ca admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:03.467949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd585f46-9bf5-4cff-a80f-c061d693d7ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:03.468637 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd585f46-9bf5-4cff-a80f-c061d693d7ff service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:03.468929 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd585f46-9bf5-4cff-a80f-c061d693d7ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:03.469331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd585f46-9bf5-4cff-a80f-c061d693d7ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:03.529801 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd585f46-9bf5-4cff-a80f-c061d693d7ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:03.531291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd585f46-9bf5-4cff-a80f-c061d693d7ff service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:03.541062 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1074/2164] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:03.799424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f9e7b-1615-44b3-883f-4a35c2c244de None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:03.799785 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f9e7b-1615-44b3-883f-4a35c2c244de None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:03.800138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f9e7b-1615-44b3-883f-4a35c2c244de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:03.819678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-713f9e7b-1615-44b3-883f-4a35c2c244de None None] MFA Rules not processed for user `34f24b9b6f2d45c4907ce8cb419ab4a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:03.821012 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-713f9e7b-1615-44b3-883f-4a35c2c244de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:03.878019 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1091/2165] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:03.996894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd585f46-9bf5-4cff-a80f-c061d693d7ff service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:03.998844 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bd585f46-9bf5-4cff-a80f-c061d693d7ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:04.070358 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd585f46-9bf5-4cff-a80f-c061d693d7ff service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:04.076056 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccdc085-000f-4b7d-8dca-a9e927897b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:04.077064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccdc085-000f-4b7d-8dca-a9e927897b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:04.077352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccdc085-000f-4b7d-8dca-a9e927897b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:04.077627 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccdc085-000f-4b7d-8dca-a9e927897b68 admin admin] PATH_INFO: `/v3/users/9959e62a4fee4385ba31455e8e76f76e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:04.078859 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccdc085-000f-4b7d-8dca-a9e927897b68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9959e62a4fee4385ba31455e8e76f76e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:04.081989 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccdc085-000f-4b7d-8dca-a9e927897b68 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:04.145668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0ccdc085-000f-4b7d-8dca-a9e927897b68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9959e62a4fee4385ba31455e8e76f76e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:04.159745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0ccdc085-000f-4b7d-8dca-a9e927897b68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9959e62a4fee4385ba31455e8e76f76e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:04.168793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0ccdc085-000f-4b7d-8dca-a9e927897b68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9959e62a4fee4385ba31455e8e76f76e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:04.169912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0ccdc085-000f-4b7d-8dca-a9e927897b68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9959e62a4fee4385ba31455e8e76f76e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:04.170544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-0ccdc085-000f-4b7d-8dca-a9e927897b68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9959e62a4fee4385ba31455e8e76f76e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:04.190138 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1075/2166] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:38:03 2022] DELETE /identity/v3/users/9959e62a4fee4385ba31455e8e76f76e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:04.194624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fad598-8a6a-40a7-8742-fe2f7c5a3f20 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:04.195162 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fad598-8a6a-40a7-8742-fe2f7c5a3f20 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:04.195351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fad598-8a6a-40a7-8742-fe2f7c5a3f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:04.213651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-c5fad598-8a6a-40a7-8742-fe2f7c5a3f20 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:04.216207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-c5fad598-8a6a-40a7-8742-fe2f7c5a3f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:04.274246 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1092/2167] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:04.284907 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccdc085-000f-4b7d-8dca-a9e927897b68 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:04.286535 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0ccdc085-000f-4b7d-8dca-a9e927897b68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:04.305994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:04.351608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccdc085-000f-4b7d-8dca-a9e927897b68 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:04.356250 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5009ed-b02c-4ef4-b08c-6b88dcd89ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:04.356911 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5009ed-b02c-4ef4-b08c-6b88dcd89ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:04.357202 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5009ed-b02c-4ef4-b08c-6b88dcd89ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:04.357489 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5009ed-b02c-4ef4-b08c-6b88dcd89ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:04.368402 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5009ed-b02c-4ef4-b08c-6b88dcd89ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:04.369979 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5009ed-b02c-4ef4-b08c-6b88dcd89ab4 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:04.388844 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1076/2168] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:04.735463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fad598-8a6a-40a7-8742-fe2f7c5a3f20 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:04.737292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-c5fad598-8a6a-40a7-8742-fe2f7c5a3f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:04.750251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fad598-8a6a-40a7-8742-fe2f7c5a3f20 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:04.756038 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1e9a4-9a1d-4ea8-8207-96b3f36750ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:04.757508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1e9a4-9a1d-4ea8-8207-96b3f36750ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:04.758076 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1e9a4-9a1d-4ea8-8207-96b3f36750ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:04.758571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c1e9a4-9a1d-4ea8-8207-96b3f36750ce admin admin] PATH_INFO: `/v3/projects/449865b12c7b4be0830db895d33ca0ae` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:04.760910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1e9a4-9a1d-4ea8-8207-96b3f36750ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=449865b12c7b4be0830db895d33ca0ae)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:04.764382 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c1e9a4-9a1d-4ea8-8207-96b3f36750ce admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:04.853000 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-f9c1e9a4-9a1d-4ea8-8207-96b3f36750ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '449865b12c7b4be0830db895d33ca0ae', 'request_id': 'req-f9c1e9a4-9a1d-4ea8-8207-96b3f36750ce'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:04.863491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-f9c1e9a4-9a1d-4ea8-8207-96b3f36750ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '449865b12c7b4be0830db895d33ca0ae', 'request_id': 'req-f9c1e9a4-9a1d-4ea8-8207-96b3f36750ce'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:04.873603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-f9c1e9a4-9a1d-4ea8-8207-96b3f36750ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '449865b12c7b4be0830db895d33ca0ae', 'request_id': 'req-f9c1e9a4-9a1d-4ea8-8207-96b3f36750ce'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:04.885303 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-f9c1e9a4-9a1d-4ea8-8207-96b3f36750ce admin admin] The token cache is being invalidate because project 449865b12c7b4be0830db895d33ca0ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:04.885601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-f9c1e9a4-9a1d-4ea8-8207-96b3f36750ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:04.886258 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-f9c1e9a4-9a1d-4ea8-8207-96b3f36750ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:04.888544 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1093/2169] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:04 2022] DELETE /identity/v3/projects/449865b12c7b4be0830db895d33ca0ae => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:04.892419 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5009ed-b02c-4ef4-b08c-6b88dcd89ab4 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:04.904386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5009ed-b02c-4ef4-b08c-6b88dcd89ab4 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:04.967773 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d11fcd-bae6-4922-9a0a-776ca11b3ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:04.968652 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d11fcd-bae6-4922-9a0a-776ca11b3ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:04.968949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d11fcd-bae6-4922-9a0a-776ca11b3ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:04.969235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d11fcd-bae6-4922-9a0a-776ca11b3ef6 admin admin] PATH_INFO: `/v3/users/5f45e6af3786488bbc8d1005a889a99f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:04.970424 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d11fcd-bae6-4922-9a0a-776ca11b3ef6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f45e6af3786488bbc8d1005a889a99f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:04.972357 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d11fcd-bae6-4922-9a0a-776ca11b3ef6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:05.021267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e4d11fcd-bae6-4922-9a0a-776ca11b3ef6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f45e6af3786488bbc8d1005a889a99f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:05.028761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e4d11fcd-bae6-4922-9a0a-776ca11b3ef6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f45e6af3786488bbc8d1005a889a99f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:05.036538 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e4d11fcd-bae6-4922-9a0a-776ca11b3ef6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f45e6af3786488bbc8d1005a889a99f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:05.037227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e4d11fcd-bae6-4922-9a0a-776ca11b3ef6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f45e6af3786488bbc8d1005a889a99f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:05.037892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-e4d11fcd-bae6-4922-9a0a-776ca11b3ef6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f45e6af3786488bbc8d1005a889a99f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:05.057012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1077/2170] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:38:04 2022] DELETE /identity/v3/users/5f45e6af3786488bbc8d1005a889a99f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:05.331451 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1e9a4-9a1d-4ea8-8207-96b3f36750ce admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:05.333074 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-f9c1e9a4-9a1d-4ea8-8207-96b3f36750ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:05.350139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c1e9a4-9a1d-4ea8-8207-96b3f36750ce admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:05.364647 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:05.411663 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5ed5c-0700-4714-bb55-f6606c06fabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:05.412733 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5ed5c-0700-4714-bb55-f6606c06fabe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:05.413104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5ed5c-0700-4714-bb55-f6606c06fabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:05.413399 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5ed5c-0700-4714-bb55-f6606c06fabe admin admin] PATH_INFO: `/v3/projects/863222117d1f4c4aa113ff37ee814a09` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:05.414601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5ed5c-0700-4714-bb55-f6606c06fabe admin admin] RBAC: Authorizing `identity:delete_project(project_id=863222117d1f4c4aa113ff37ee814a09)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:05.416588 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5ed5c-0700-4714-bb55-f6606c06fabe admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:05.463811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e1b5ed5c-0700-4714-bb55-f6606c06fabe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '863222117d1f4c4aa113ff37ee814a09', 'request_id': 'req-e1b5ed5c-0700-4714-bb55-f6606c06fabe'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:05.471371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e1b5ed5c-0700-4714-bb55-f6606c06fabe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '863222117d1f4c4aa113ff37ee814a09', 'request_id': 'req-e1b5ed5c-0700-4714-bb55-f6606c06fabe'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:05.479609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e1b5ed5c-0700-4714-bb55-f6606c06fabe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '863222117d1f4c4aa113ff37ee814a09', 'request_id': 'req-e1b5ed5c-0700-4714-bb55-f6606c06fabe'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:05.491256 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e1b5ed5c-0700-4714-bb55-f6606c06fabe admin admin] The token cache is being invalidate because project 863222117d1f4c4aa113ff37ee814a09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:05.491730 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e1b5ed5c-0700-4714-bb55-f6606c06fabe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:05.492390 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e1b5ed5c-0700-4714-bb55-f6606c06fabe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:05.494135 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1094/2171] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:05 2022] DELETE /identity/v3/projects/863222117d1f4c4aa113ff37ee814a09 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:05.498433 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d11fcd-bae6-4922-9a0a-776ca11b3ef6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:05.499887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e4d11fcd-bae6-4922-9a0a-776ca11b3ef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:05.514564 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d11fcd-bae6-4922-9a0a-776ca11b3ef6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:05.550297 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:05.573706 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70374937-5b9a-4b95-8c58-d62ecaa25852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:05.575156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70374937-5b9a-4b95-8c58-d62ecaa25852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:05.575440 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70374937-5b9a-4b95-8c58-d62ecaa25852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:05.575716 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70374937-5b9a-4b95-8c58-d62ecaa25852 admin admin] PATH_INFO: `/v3/users/04274c6585644f36aeadf89b6c669bd4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:05.576891 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70374937-5b9a-4b95-8c58-d62ecaa25852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04274c6585644f36aeadf89b6c669bd4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:05.578795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70374937-5b9a-4b95-8c58-d62ecaa25852 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:05.627732 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-70374937-5b9a-4b95-8c58-d62ecaa25852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04274c6585644f36aeadf89b6c669bd4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:05.634801 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-70374937-5b9a-4b95-8c58-d62ecaa25852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04274c6585644f36aeadf89b6c669bd4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:05.643465 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-70374937-5b9a-4b95-8c58-d62ecaa25852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04274c6585644f36aeadf89b6c669bd4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:05.644327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-70374937-5b9a-4b95-8c58-d62ecaa25852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04274c6585644f36aeadf89b6c669bd4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:05.644891 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-70374937-5b9a-4b95-8c58-d62ecaa25852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04274c6585644f36aeadf89b6c669bd4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:05.665342 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1078/2172] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:38:05 2022] DELETE /identity/v3/users/04274c6585644f36aeadf89b6c669bd4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:05.945306 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5ed5c-0700-4714-bb55-f6606c06fabe admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:05.946831 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e1b5ed5c-0700-4714-bb55-f6606c06fabe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:05.963725 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5ed5c-0700-4714-bb55-f6606c06fabe admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:06.026327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0334b-c1c1-4bf7-82db-86feaa9c6f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:06.027578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0334b-c1c1-4bf7-82db-86feaa9c6f53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:06.027829 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0334b-c1c1-4bf7-82db-86feaa9c6f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:06.028038 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0334b-c1c1-4bf7-82db-86feaa9c6f53 admin admin] PATH_INFO: `/v3/projects/d17e57f09e7c42ca8dd90f1b9a79e79e` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:06.029334 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0334b-c1c1-4bf7-82db-86feaa9c6f53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d17e57f09e7c42ca8dd90f1b9a79e79e)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:06.031387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0334b-c1c1-4bf7-82db-86feaa9c6f53 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:06.079714 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-fdb0334b-c1c1-4bf7-82db-86feaa9c6f53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd17e57f09e7c42ca8dd90f1b9a79e79e', 'request_id': 'req-fdb0334b-c1c1-4bf7-82db-86feaa9c6f53'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:06.080820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:06.088179 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-fdb0334b-c1c1-4bf7-82db-86feaa9c6f53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd17e57f09e7c42ca8dd90f1b9a79e79e', 'request_id': 'req-fdb0334b-c1c1-4bf7-82db-86feaa9c6f53'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:06.096500 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-fdb0334b-c1c1-4bf7-82db-86feaa9c6f53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd17e57f09e7c42ca8dd90f1b9a79e79e', 'request_id': 'req-fdb0334b-c1c1-4bf7-82db-86feaa9c6f53'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:06.115207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-fdb0334b-c1c1-4bf7-82db-86feaa9c6f53 admin admin] The token cache is being invalidate because project d17e57f09e7c42ca8dd90f1b9a79e79e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:06.115529 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-fdb0334b-c1c1-4bf7-82db-86feaa9c6f53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:06.116189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-fdb0334b-c1c1-4bf7-82db-86feaa9c6f53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:06.117734 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1095/2173] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:05 2022] DELETE /identity/v3/projects/d17e57f09e7c42ca8dd90f1b9a79e79e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:06.123665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b995e8-4e1e-4620-88ee-7b5373a3fbbe None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:06.124141 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b995e8-4e1e-4620-88ee-7b5373a3fbbe None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:06.124252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b995e8-4e1e-4620-88ee-7b5373a3fbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:06.143418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-69b995e8-4e1e-4620-88ee-7b5373a3fbbe None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:06.144624 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-69b995e8-4e1e-4620-88ee-7b5373a3fbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:06.205210 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1079/2174] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:06.212654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d326c6-b911-46b7-a735-749734e9cb20 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:06.212937 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d326c6-b911-46b7-a735-749734e9cb20 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:06.213218 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d326c6-b911-46b7-a735-749734e9cb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:06.232241 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-a8d326c6-b911-46b7-a735-749734e9cb20 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:06.233426 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-a8d326c6-b911-46b7-a735-749734e9cb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:06.241215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1096/2175] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:06.245302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b995e8-4e1e-4620-88ee-7b5373a3fbbe None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:06.256170 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b995e8-4e1e-4620-88ee-7b5373a3fbbe None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:06.260686 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b02210-7d8e-46d0-be16-32c84a793abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:06.261338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b02210-7d8e-46d0-be16-32c84a793abe admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:06.261628 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b02210-7d8e-46d0-be16-32c84a793abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:06.262499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b02210-7d8e-46d0-be16-32c84a793abe admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:06.263299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b02210-7d8e-46d0-be16-32c84a793abe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:38:06.263613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b02210-7d8e-46d0-be16-32c84a793abe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:06.265242 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b02210-7d8e-46d0-be16-32c84a793abe admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:06.276882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1080/2176] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:38:06.280931 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d326c6-b911-46b7-a735-749734e9cb20 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:06.290377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d326c6-b911-46b7-a735-749734e9cb20 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:06.296042 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737b4ca-fc76-4569-aee4-88ab8b84cfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:06.296689 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737b4ca-fc76-4569-aee4-88ab8b84cfdb admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:06.296976 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737b4ca-fc76-4569-aee4-88ab8b84cfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:06.297256 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737b4ca-fc76-4569-aee4-88ab8b84cfdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:06.297960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737b4ca-fc76-4569-aee4-88ab8b84cfdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:06.299898 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737b4ca-fc76-4569-aee4-88ab8b84cfdb admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:06.325873 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1097/2177] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:38:06 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:06.329871 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b02210-7d8e-46d0-be16-32c84a793abe admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:06.339644 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b02210-7d8e-46d0-be16-32c84a793abe admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:06.381201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:06.397440 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9426821-d80c-4adb-8267-8dd674d4fadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:06.398151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9426821-d80c-4adb-8267-8dd674d4fadb admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:06.398442 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9426821-d80c-4adb-8267-8dd674d4fadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:06.398720 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9426821-d80c-4adb-8267-8dd674d4fadb admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:06.399454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9426821-d80c-4adb-8267-8dd674d4fadb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:06.401357 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9426821-d80c-4adb-8267-8dd674d4fadb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:06.421152 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1081/2178] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:38:06 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:38:06.425232 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737b4ca-fc76-4569-aee4-88ab8b84cfdb admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:06.436623 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737b4ca-fc76-4569-aee4-88ab8b84cfdb admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:06.441256 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbeb439-f994-42bb-9c80-1ff8e3e63846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:06.441939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbeb439-f994-42bb-9c80-1ff8e3e63846 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:06.442227 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbeb439-f994-42bb-9c80-1ff8e3e63846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:06.442508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbeb439-f994-42bb-9c80-1ff8e3e63846 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:06.443265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbeb439-f994-42bb-9c80-1ff8e3e63846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:06.444868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbeb439-f994-42bb-9c80-1ff8e3e63846 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:06.455571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1098/2179] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:06.459563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9426821-d80c-4adb-8267-8dd674d4fadb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:06.469799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9426821-d80c-4adb-8267-8dd674d4fadb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:06.474315 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446aff0-1316-4c51-9f68-95b489948a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:06.474992 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446aff0-1316-4c51-9f68-95b489948a69 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:06.475291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446aff0-1316-4c51-9f68-95b489948a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:06.475555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446aff0-1316-4c51-9f68-95b489948a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:06.476278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446aff0-1316-4c51-9f68-95b489948a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:06.477849 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446aff0-1316-4c51-9f68-95b489948a69 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:06.488394 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1082/2180] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:06.492322 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbeb439-f994-42bb-9c80-1ff8e3e63846 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:06.502587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbeb439-f994-42bb-9c80-1ff8e3e63846 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:06.507290 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d880ea7-c98b-415a-85d8-96eeb8726294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:06.508354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d880ea7-c98b-415a-85d8-96eeb8726294 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:06.508650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d880ea7-c98b-415a-85d8-96eeb8726294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:06.508935 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d880ea7-c98b-415a-85d8-96eeb8726294 admin admin] PATH_INFO: `/v3/projects/30f69128017743eea63a05b325a2e246/users/03f4d15861a84b24937edfa81221fc7b/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:06.523330 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d880ea7-c98b-415a-85d8-96eeb8726294 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f69128017743eea63a05b325a2e246, user_id=03f4d15861a84b24937edfa81221fc7b, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:06.525961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d880ea7-c98b-415a-85d8-96eeb8726294 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:06.539187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1099/2181] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:06 2022] PUT /identity/v3/projects/30f69128017743eea63a05b325a2e246/users/03f4d15861a84b24937edfa81221fc7b/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:06.550862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704e7e8-ef10-4ac2-9da5-5a2160d6980c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:06.551187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704e7e8-ef10-4ac2-9da5-5a2160d6980c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:06.551468 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0704e7e8-ef10-4ac2-9da5-5a2160d6980c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:06.601084 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0704e7e8-ef10-4ac2-9da5-5a2160d6980c None None] MFA Rules not processed for user `03f4d15861a84b24937edfa81221fc7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:06.602537 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0704e7e8-ef10-4ac2-9da5-5a2160d6980c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:06.708314 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1083/2182] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:06.711612 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d880ea7-c98b-415a-85d8-96eeb8726294 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:06.779671 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d880ea7-c98b-415a-85d8-96eeb8726294 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:06.784247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089f183-8a52-42bc-aa0a-9cf11177b49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:06.785180 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089f183-8a52-42bc-aa0a-9cf11177b49d admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:06.785456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089f183-8a52-42bc-aa0a-9cf11177b49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:06.785899 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089f183-8a52-42bc-aa0a-9cf11177b49d admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:06.786596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089f183-8a52-42bc-aa0a-9cf11177b49d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:06.788545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089f183-8a52-42bc-aa0a-9cf11177b49d admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:06.821796 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1100/2183] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:38:06 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:06.826667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704e7e8-ef10-4ac2-9da5-5a2160d6980c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:06.837786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0704e7e8-ef10-4ac2-9da5-5a2160d6980c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:06.897212 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab70f5c-2e32-4cce-be03-2953def6f065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:06.897946 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab70f5c-2e32-4cce-be03-2953def6f065 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:06.898182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab70f5c-2e32-4cce-be03-2953def6f065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:06.898466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab70f5c-2e32-4cce-be03-2953def6f065 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:06.899206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab70f5c-2e32-4cce-be03-2953def6f065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:06.901099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab70f5c-2e32-4cce-be03-2953def6f065 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:06.922330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1084/2184] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:38:06 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:06.926503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089f183-8a52-42bc-aa0a-9cf11177b49d admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:06.937646 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089f183-8a52-42bc-aa0a-9cf11177b49d admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:06.942312 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc63fd-ea30-4d1a-978a-149b93120110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:06.942998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc63fd-ea30-4d1a-978a-149b93120110 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:06.943285 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc63fd-ea30-4d1a-978a-149b93120110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:06.943565 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc63fd-ea30-4d1a-978a-149b93120110 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:06.944288 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc63fd-ea30-4d1a-978a-149b93120110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:06.945876 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc63fd-ea30-4d1a-978a-149b93120110 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:06.956297 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1101/2185] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:06.960299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab70f5c-2e32-4cce-be03-2953def6f065 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:06.970287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab70f5c-2e32-4cce-be03-2953def6f065 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:06.974864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86ab3b-33b6-4260-961b-b11f7ecd5900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:06.975558 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86ab3b-33b6-4260-961b-b11f7ecd5900 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:06.975847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86ab3b-33b6-4260-961b-b11f7ecd5900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:06.976125 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86ab3b-33b6-4260-961b-b11f7ecd5900 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:06.976849 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86ab3b-33b6-4260-961b-b11f7ecd5900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:06.978410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86ab3b-33b6-4260-961b-b11f7ecd5900 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:06.990956 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1085/2186] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:06.993656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc63fd-ea30-4d1a-978a-149b93120110 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:07.004850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc63fd-ea30-4d1a-978a-149b93120110 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:07.010253 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df06a3-d6a7-4939-ac58-4e11f9e0c197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:07.011178 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df06a3-d6a7-4939-ac58-4e11f9e0c197 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:07.011465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df06a3-d6a7-4939-ac58-4e11f9e0c197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:07.011791 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df06a3-d6a7-4939-ac58-4e11f9e0c197 admin admin] PATH_INFO: `/v3/projects/f687637bd4164671890f3f90b66fd78c/users/1a4e434eec5b4f1da9ed8dfab927f63d/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:07.026465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df06a3-d6a7-4939-ac58-4e11f9e0c197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f687637bd4164671890f3f90b66fd78c, user_id=1a4e434eec5b4f1da9ed8dfab927f63d, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:07.029803 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df06a3-d6a7-4939-ac58-4e11f9e0c197 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:07.042409 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1102/2187] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:06 2022] PUT /identity/v3/projects/f687637bd4164671890f3f90b66fd78c/users/1a4e434eec5b4f1da9ed8dfab927f63d/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:38:07.053628 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84582f22-6d6b-4ad9-9f8a-c11c658c44fe None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:07.054143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84582f22-6d6b-4ad9-9f8a-c11c658c44fe None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:07.054441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84582f22-6d6b-4ad9-9f8a-c11c658c44fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:07.081485 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-84582f22-6d6b-4ad9-9f8a-c11c658c44fe None None] MFA Rules not processed for user `1a4e434eec5b4f1da9ed8dfab927f63d`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:07.082728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-84582f22-6d6b-4ad9-9f8a-c11c658c44fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:07.100974 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:07.149730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1086/2188] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:07.154245 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df06a3-d6a7-4939-ac58-4e11f9e0c197 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:07.181785 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:07.219761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df06a3-d6a7-4939-ac58-4e11f9e0c197 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:07.224985 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78958044-ae09-4d53-bab3-f75b0d0d01c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:07.225650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78958044-ae09-4d53-bab3-f75b0d0d01c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:07.225935 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78958044-ae09-4d53-bab3-f75b0d0d01c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:07.226327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78958044-ae09-4d53-bab3-f75b0d0d01c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:07.226932 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78958044-ae09-4d53-bab3-f75b0d0d01c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:07.228896 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78958044-ae09-4d53-bab3-f75b0d0d01c9 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:07.261446 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1103/2189] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:38:07 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:38:07.262907 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84582f22-6d6b-4ad9-9f8a-c11c658c44fe None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:07.274058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84582f22-6d6b-4ad9-9f8a-c11c658c44fe None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:07.331239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7b947-1353-42d9-9dbe-f2bfeb4e6ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:07.331919 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7b947-1353-42d9-9dbe-f2bfeb4e6ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:07.332214 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7b947-1353-42d9-9dbe-f2bfeb4e6ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:07.332503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7b947-1353-42d9-9dbe-f2bfeb4e6ae7 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:07.333194 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7b947-1353-42d9-9dbe-f2bfeb4e6ae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:07.335146 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7b947-1353-42d9-9dbe-f2bfeb4e6ae7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:07.356388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1087/2190] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:38:07 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:38:07.360900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78958044-ae09-4d53-bab3-f75b0d0d01c9 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:07.370996 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78958044-ae09-4d53-bab3-f75b0d0d01c9 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:07.375625 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc7a2e-7cc5-4d87-bc4e-9dad3113e869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:07.376278 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc7a2e-7cc5-4d87-bc4e-9dad3113e869 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:07.376562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc7a2e-7cc5-4d87-bc4e-9dad3113e869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:07.376841 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc7a2e-7cc5-4d87-bc4e-9dad3113e869 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:07.377592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc7a2e-7cc5-4d87-bc4e-9dad3113e869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:07.379228 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc7a2e-7cc5-4d87-bc4e-9dad3113e869 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:07.389925 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1104/2191] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:07.393832 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7b947-1353-42d9-9dbe-f2bfeb4e6ae7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:07.404341 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7b947-1353-42d9-9dbe-f2bfeb4e6ae7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:07.408999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc2523-d4e2-45fc-85c2-16ccf492ecf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:07.410159 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc2523-d4e2-45fc-85c2-16ccf492ecf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:07.410447 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc2523-d4e2-45fc-85c2-16ccf492ecf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:07.410726 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc2523-d4e2-45fc-85c2-16ccf492ecf0 admin admin] PATH_INFO: `/v3/projects/e8831a73e2e1432eb1c2719b54b3a512/users/74782852e2b5408e99e7b26acfbd2d8f/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:07.425555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc2523-d4e2-45fc-85c2-16ccf492ecf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8831a73e2e1432eb1c2719b54b3a512, user_id=74782852e2b5408e99e7b26acfbd2d8f, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:07.428224 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc2523-d4e2-45fc-85c2-16ccf492ecf0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:07.440985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1088/2192] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:07 2022] PUT /identity/v3/projects/e8831a73e2e1432eb1c2719b54b3a512/users/74782852e2b5408e99e7b26acfbd2d8f/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:38:07.444811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc7a2e-7cc5-4d87-bc4e-9dad3113e869 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:07.454968 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc7a2e-7cc5-4d87-bc4e-9dad3113e869 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:07.511595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31462ba7-39a3-44c0-8a46-a25248e2b42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:07.512345 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31462ba7-39a3-44c0-8a46-a25248e2b42e admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:07.512631 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31462ba7-39a3-44c0-8a46-a25248e2b42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:07.512909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31462ba7-39a3-44c0-8a46-a25248e2b42e admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:07.513625 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31462ba7-39a3-44c0-8a46-a25248e2b42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:07.515245 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31462ba7-39a3-44c0-8a46-a25248e2b42e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:07.525030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1105/2193] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:07.529678 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc2523-d4e2-45fc-85c2-16ccf492ecf0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:07.540193 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc2523-d4e2-45fc-85c2-16ccf492ecf0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:07.545722 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5452d0-44cf-47d8-bd72-05219d71b995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:07.546599 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5452d0-44cf-47d8-bd72-05219d71b995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:07.546887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5452d0-44cf-47d8-bd72-05219d71b995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:07.547197 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5452d0-44cf-47d8-bd72-05219d71b995 admin admin] PATH_INFO: `/v3/projects/e8831a73e2e1432eb1c2719b54b3a512/users/74782852e2b5408e99e7b26acfbd2d8f/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:07.549137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5452d0-44cf-47d8-bd72-05219d71b995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8831a73e2e1432eb1c2719b54b3a512, user_id=74782852e2b5408e99e7b26acfbd2d8f, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:07.551781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5452d0-44cf-47d8-bd72-05219d71b995 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:07.562071 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1089/2194] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:07 2022] PUT /identity/v3/projects/e8831a73e2e1432eb1c2719b54b3a512/users/74782852e2b5408e99e7b26acfbd2d8f/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:07.572124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14927533-fdbd-4258-824f-eef2d97e6a44 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:07.572421 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14927533-fdbd-4258-824f-eef2d97e6a44 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:07.572705 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14927533-fdbd-4258-824f-eef2d97e6a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:07.599424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-14927533-fdbd-4258-824f-eef2d97e6a44 None None] MFA Rules not processed for user `74782852e2b5408e99e7b26acfbd2d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:07.600618 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-14927533-fdbd-4258-824f-eef2d97e6a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:07.673506 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1106/2195] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:07.684797 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5452d0-44cf-47d8-bd72-05219d71b995 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:07.686592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2c5452d0-44cf-47d8-bd72-05219d71b995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:07.749400 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5452d0-44cf-47d8-bd72-05219d71b995 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:07.753166 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb6a4b-bb18-482f-af20-1067e6bb89f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:07.753827 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb6a4b-bb18-482f-af20-1067e6bb89f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:07.754116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb6a4b-bb18-482f-af20-1067e6bb89f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:07.754535 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb6a4b-bb18-482f-af20-1067e6bb89f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:07.789681 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:07.820365 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb6a4b-bb18-482f-af20-1067e6bb89f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:07.821799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb6a4b-bb18-482f-af20-1067e6bb89f0 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:07.831788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1090/2196] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:11.066910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2531e-1cf6-4fe8-932a-86c386cd892c None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:11.068218 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2531e-1cf6-4fe8-932a-86c386cd892c None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:11.068569 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2531e-1cf6-4fe8-932a-86c386cd892c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:11.095053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-2ff2531e-1cf6-4fe8-932a-86c386cd892c None None] MFA Rules not processed for user `df1aeb2be41c49819b811ecfa47d4155`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:11.096433 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2ff2531e-1cf6-4fe8-932a-86c386cd892c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:11.114715 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:11.178134 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1107/2197] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 22:38:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:11.249180 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb6a4b-bb18-482f-af20-1067e6bb89f0 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:11.251842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fcdb6a4b-bb18-482f-af20-1067e6bb89f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:11.404537 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb6a4b-bb18-482f-af20-1067e6bb89f0 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:11.422351 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35dcacf-b2c8-47af-bad4-f9200ca92a55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ce35c43e8294f4cbc880f2219a6eb76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:11.424366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35dcacf-b2c8-47af-bad4-f9200ca92a55 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:11.425518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35dcacf-b2c8-47af-bad4-f9200ca92a55 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:11.426161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35dcacf-b2c8-47af-bad4-f9200ca92a55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:11.451955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35dcacf-b2c8-47af-bad4-f9200ca92a55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:11.453701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35dcacf-b2c8-47af-bad4-f9200ca92a55 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:11.482258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1091/2198] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 22:38:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:38:11.678568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2531e-1cf6-4fe8-932a-86c386cd892c None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:11.681369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2ff2531e-1cf6-4fe8-932a-86c386cd892c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:11.711603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2531e-1cf6-4fe8-932a-86c386cd892c None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:11.735082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae53170-ee58-4898-8dfb-584fd7868826 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:11.736078 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae53170-ee58-4898-8dfb-584fd7868826 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:11.736940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae53170-ee58-4898-8dfb-584fd7868826 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:11.737382 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae53170-ee58-4898-8dfb-584fd7868826 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:11.826871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae53170-ee58-4898-8dfb-584fd7868826 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:11.829235 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae53170-ee58-4898-8dfb-584fd7868826 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:11.839759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1108/2199] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:13.251396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35dcacf-b2c8-47af-bad4-f9200ca92a55 service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:13.264880 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35dcacf-b2c8-47af-bad4-f9200ca92a55 service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:13.269331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ebb6d-4fff-46a3-b0cf-c9ef6fe5f77b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:13.270274 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ebb6d-4fff-46a3-b0cf-c9ef6fe5f77b service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:13.270795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ebb6d-4fff-46a3-b0cf-c9ef6fe5f77b service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:13.271368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ebb6d-4fff-46a3-b0cf-c9ef6fe5f77b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:13.284620 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ebb6d-4fff-46a3-b0cf-c9ef6fe5f77b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:13.286634 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ebb6d-4fff-46a3-b0cf-c9ef6fe5f77b service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:13.298218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1092/2200] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:15.968428 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae53170-ee58-4898-8dfb-584fd7868826 service cinder] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:15.985882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae53170-ee58-4898-8dfb-584fd7868826 service cinder] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:15.992784 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8991a-f9da-4986-a3df-be24cd6b2970 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:15.994156 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8991a-f9da-4986-a3df-be24cd6b2970 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:15.994667 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8991a-f9da-4986-a3df-be24cd6b2970 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:15.996020 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8991a-f9da-4986-a3df-be24cd6b2970 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:16.000589 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-fbf8991a-f9da-4986-a3df-be24cd6b2970 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:16.098453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ebb6d-4fff-46a3-b0cf-c9ef6fe5f77b service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:16.101099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-632ebb6d-4fff-46a3-b0cf-c9ef6fe5f77b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:16.181847 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8991a-f9da-4986-a3df-be24cd6b2970 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:16.183524 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8991a-f9da-4986-a3df-be24cd6b2970 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:16.212755 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1109/2201] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:15 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:16.333576 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ebb6d-4fff-46a3-b0cf-c9ef6fe5f77b service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:16.342883 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ad07e-2f82-4496-a533-b2d946a8eb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:16.345037 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ad07e-2f82-4496-a533-b2d946a8eb63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:16.345470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ad07e-2f82-4496-a533-b2d946a8eb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:16.345901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ad07e-2f82-4496-a533-b2d946a8eb63 admin admin] PATH_INFO: `/v3/users/b15d667a30ee40beb7aa2affcb490fc8` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:16.348491 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ad07e-2f82-4496-a533-b2d946a8eb63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b15d667a30ee40beb7aa2affcb490fc8)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:16.352408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ad07e-2f82-4496-a533-b2d946a8eb63 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:16.458713 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:16.463291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fa4ad07e-2f82-4496-a533-b2d946a8eb63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b15d667a30ee40beb7aa2affcb490fc8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:16.487673 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fa4ad07e-2f82-4496-a533-b2d946a8eb63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b15d667a30ee40beb7aa2affcb490fc8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:16.498825 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fa4ad07e-2f82-4496-a533-b2d946a8eb63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b15d667a30ee40beb7aa2affcb490fc8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:16.499510 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fa4ad07e-2f82-4496-a533-b2d946a8eb63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b15d667a30ee40beb7aa2affcb490fc8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:16.500179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fa4ad07e-2f82-4496-a533-b2d946a8eb63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b15d667a30ee40beb7aa2affcb490fc8'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:16.538624 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1093/2202] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:38:16 2022] DELETE /identity/v3/users/b15d667a30ee40beb7aa2affcb490fc8 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:16.543781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cee3cb-69a2-4540-89a9-06d1bb733be4 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:16.544077 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cee3cb-69a2-4540-89a9-06d1bb733be4 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:16.544364 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cee3cb-69a2-4540-89a9-06d1bb733be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:16.568583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-30cee3cb-69a2-4540-89a9-06d1bb733be4 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:16.571096 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-30cee3cb-69a2-4540-89a9-06d1bb733be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:16.646604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:16.664435 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1110/2203] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:16.681021 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ad07e-2f82-4496-a533-b2d946a8eb63 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:16.682791 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fa4ad07e-2f82-4496-a533-b2d946a8eb63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:16.809069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ad07e-2f82-4496-a533-b2d946a8eb63 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:16.816362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fa83f-a0af-45bc-8233-8311826de4f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:16.817273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fa83f-a0af-45bc-8233-8311826de4f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:16.817632 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fa83f-a0af-45bc-8233-8311826de4f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:16.818607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fa83f-a0af-45bc-8233-8311826de4f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:16.839746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5fa83f-a0af-45bc-8233-8311826de4f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:16.844681 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5fa83f-a0af-45bc-8233-8311826de4f5 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:16.865786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1094/2204] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:17.262859 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cee3cb-69a2-4540-89a9-06d1bb733be4 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:17.269212 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-30cee3cb-69a2-4540-89a9-06d1bb733be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:17.290683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cee3cb-69a2-4540-89a9-06d1bb733be4 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:17.298501 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480dcb89-66dd-4d67-8240-4c9a87ba60c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:17.300046 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480dcb89-66dd-4d67-8240-4c9a87ba60c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:17.300568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480dcb89-66dd-4d67-8240-4c9a87ba60c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:17.301106 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480dcb89-66dd-4d67-8240-4c9a87ba60c7 admin admin] PATH_INFO: `/v3/projects/052e490afbed45a783fd7adc1e08823f` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:17.303142 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480dcb89-66dd-4d67-8240-4c9a87ba60c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=052e490afbed45a783fd7adc1e08823f)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:17.307327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480dcb89-66dd-4d67-8240-4c9a87ba60c7 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:17.424027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-480dcb89-66dd-4d67-8240-4c9a87ba60c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '052e490afbed45a783fd7adc1e08823f', 'request_id': 'req-480dcb89-66dd-4d67-8240-4c9a87ba60c7'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:17.433308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-480dcb89-66dd-4d67-8240-4c9a87ba60c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '052e490afbed45a783fd7adc1e08823f', 'request_id': 'req-480dcb89-66dd-4d67-8240-4c9a87ba60c7'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:17.460136 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-480dcb89-66dd-4d67-8240-4c9a87ba60c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '052e490afbed45a783fd7adc1e08823f', 'request_id': 'req-480dcb89-66dd-4d67-8240-4c9a87ba60c7'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:17.494119 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-480dcb89-66dd-4d67-8240-4c9a87ba60c7 admin admin] The token cache is being invalidate because project 052e490afbed45a783fd7adc1e08823f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:17.494482 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-480dcb89-66dd-4d67-8240-4c9a87ba60c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:17.495396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-480dcb89-66dd-4d67-8240-4c9a87ba60c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:17.498593 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1111/2205] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:17 2022] DELETE /identity/v3/projects/052e490afbed45a783fd7adc1e08823f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:17.503449 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee786e71-16fd-41b8-b587-c2e47acccb95 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:17.503970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee786e71-16fd-41b8-b587-c2e47acccb95 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:17.504115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee786e71-16fd-41b8-b587-c2e47acccb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:17.547540 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ee786e71-16fd-41b8-b587-c2e47acccb95 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:17.558042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ee786e71-16fd-41b8-b587-c2e47acccb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:17.695085 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1095/2206] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:17.705971 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a0fa2-ec20-4c67-a620-40ea5f517556 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:17.706264 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a0fa2-ec20-4c67-a620-40ea5f517556 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:17.706553 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a0fa2-ec20-4c67-a620-40ea5f517556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:17.743715 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-122a0fa2-ec20-4c67-a620-40ea5f517556 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:17.744956 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-122a0fa2-ec20-4c67-a620-40ea5f517556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:17.755757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1112/2207] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:17.759803 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee786e71-16fd-41b8-b587-c2e47acccb95 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:17.774099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee786e71-16fd-41b8-b587-c2e47acccb95 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:17.785967 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c9eb5-4a88-4459-b016-d21088a6e4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:17.786910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c9eb5-4a88-4459-b016-d21088a6e4f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:17.787387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c9eb5-4a88-4459-b016-d21088a6e4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:17.787983 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c9eb5-4a88-4459-b016-d21088a6e4f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:17.788873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c9eb5-4a88-4459-b016-d21088a6e4f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:38:17.789397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c9eb5-4a88-4459-b016-d21088a6e4f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:17.791987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c9eb5-4a88-4459-b016-d21088a6e4f9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:17.811023 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1096/2208] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:38:17.815154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a0fa2-ec20-4c67-a620-40ea5f517556 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:17.827808 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a0fa2-ec20-4c67-a620-40ea5f517556 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:17.832885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850dcb5-9b4c-4ff3-9d2b-be9228d1ab29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:17.833712 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850dcb5-9b4c-4ff3-9d2b-be9228d1ab29 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:17.834049 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850dcb5-9b4c-4ff3-9d2b-be9228d1ab29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:17.834275 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850dcb5-9b4c-4ff3-9d2b-be9228d1ab29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:17.835148 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850dcb5-9b4c-4ff3-9d2b-be9228d1ab29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:17.837060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850dcb5-9b4c-4ff3-9d2b-be9228d1ab29 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:17.872713 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1113/2209] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:38:17 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:38:17.873918 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c9eb5-4a88-4459-b016-d21088a6e4f9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:17.885087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c9eb5-4a88-4459-b016-d21088a6e4f9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:17.953915 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:17.967296 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af890a0-7ed6-41e1-992e-1896c061ffd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:17.968454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af890a0-7ed6-41e1-992e-1896c061ffd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:17.968826 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af890a0-7ed6-41e1-992e-1896c061ffd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:17.969370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af890a0-7ed6-41e1-992e-1896c061ffd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:17.970488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af890a0-7ed6-41e1-992e-1896c061ffd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:17.972766 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af890a0-7ed6-41e1-992e-1896c061ffd2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:17.996564 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1097/2210] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:38:17 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:18.002470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850dcb5-9b4c-4ff3-9d2b-be9228d1ab29 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:18.013602 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850dcb5-9b4c-4ff3-9d2b-be9228d1ab29 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:18.018286 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b05e62-9bfc-4830-9dd8-9255d218b726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:18.018971 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b05e62-9bfc-4830-9dd8-9255d218b726 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:18.019263 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b05e62-9bfc-4830-9dd8-9255d218b726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:18.019548 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b05e62-9bfc-4830-9dd8-9255d218b726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:18.020313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b05e62-9bfc-4830-9dd8-9255d218b726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:18.021917 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b05e62-9bfc-4830-9dd8-9255d218b726 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:18.032454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1114/2211] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:18.036676 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af890a0-7ed6-41e1-992e-1896c061ffd2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:18.046819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af890a0-7ed6-41e1-992e-1896c061ffd2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:18.051658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16a415-5bdc-4e7e-aaad-9b63947aa294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:18.052356 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16a415-5bdc-4e7e-aaad-9b63947aa294 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:18.052647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16a415-5bdc-4e7e-aaad-9b63947aa294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:18.052932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16a415-5bdc-4e7e-aaad-9b63947aa294 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:18.053664 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16a415-5bdc-4e7e-aaad-9b63947aa294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:18.055366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16a415-5bdc-4e7e-aaad-9b63947aa294 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:18.066387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1098/2212] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:18.070410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b05e62-9bfc-4830-9dd8-9255d218b726 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:18.080979 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b05e62-9bfc-4830-9dd8-9255d218b726 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:18.085640 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d7aa0-25b7-4a73-829a-4ba85e175324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:18.086549 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d7aa0-25b7-4a73-829a-4ba85e175324 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:18.086841 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d7aa0-25b7-4a73-829a-4ba85e175324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:18.087155 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d7aa0-25b7-4a73-829a-4ba85e175324 admin admin] PATH_INFO: `/v3/projects/1eb4b659032e4ec09a16c45b29d609f0/users/f523b529906f4fa989d0e836d6a08944/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:18.101927 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d7aa0-25b7-4a73-829a-4ba85e175324 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eb4b659032e4ec09a16c45b29d609f0, user_id=f523b529906f4fa989d0e836d6a08944, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:18.104631 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d7aa0-25b7-4a73-829a-4ba85e175324 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:18.118336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1115/2213] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:18 2022] PUT /identity/v3/projects/1eb4b659032e4ec09a16c45b29d609f0/users/f523b529906f4fa989d0e836d6a08944/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:18.124163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68deef3b-8c21-4ca4-a7f8-e8a015a4ec9e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:18.124462 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68deef3b-8c21-4ca4-a7f8-e8a015a4ec9e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:18.124749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68deef3b-8c21-4ca4-a7f8-e8a015a4ec9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:18.148017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-68deef3b-8c21-4ca4-a7f8-e8a015a4ec9e None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:18.149238 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-68deef3b-8c21-4ca4-a7f8-e8a015a4ec9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:18.208069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1099/2214] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:18.217591 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d7aa0-25b7-4a73-829a-4ba85e175324 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:18.219305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-853d7aa0-25b7-4a73-829a-4ba85e175324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:18.275557 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d7aa0-25b7-4a73-829a-4ba85e175324 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:18.279803 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959f1aa-aeb9-47d6-b21e-2681cdd056a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:18.280790 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959f1aa-aeb9-47d6-b21e-2681cdd056a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:18.281093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959f1aa-aeb9-47d6-b21e-2681cdd056a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:18.281441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959f1aa-aeb9-47d6-b21e-2681cdd056a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:18.291934 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959f1aa-aeb9-47d6-b21e-2681cdd056a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:18.294392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959f1aa-aeb9-47d6-b21e-2681cdd056a8 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:18.307685 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1116/2215] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:21.166778 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68deef3b-8c21-4ca4-a7f8-e8a015a4ec9e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:21.170120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-68deef3b-8c21-4ca4-a7f8-e8a015a4ec9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:21.203480 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68deef3b-8c21-4ca4-a7f8-e8a015a4ec9e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:21.217902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be343c2-f317-4915-85f2-69ff82099a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:21.219628 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be343c2-f317-4915-85f2-69ff82099a9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:21.220066 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be343c2-f317-4915-85f2-69ff82099a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:21.220483 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be343c2-f317-4915-85f2-69ff82099a9b admin admin] PATH_INFO: `/v3/users/03f4d15861a84b24937edfa81221fc7b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:21.222169 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be343c2-f317-4915-85f2-69ff82099a9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=03f4d15861a84b24937edfa81221fc7b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:21.225143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be343c2-f317-4915-85f2-69ff82099a9b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:21.370217 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1be343c2-f317-4915-85f2-69ff82099a9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03f4d15861a84b24937edfa81221fc7b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:21.389149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1be343c2-f317-4915-85f2-69ff82099a9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03f4d15861a84b24937edfa81221fc7b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:21.404042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1be343c2-f317-4915-85f2-69ff82099a9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03f4d15861a84b24937edfa81221fc7b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:21.405965 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1be343c2-f317-4915-85f2-69ff82099a9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03f4d15861a84b24937edfa81221fc7b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:21.414707 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1be343c2-f317-4915-85f2-69ff82099a9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03f4d15861a84b24937edfa81221fc7b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:21.450027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1100/2216] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:38:21 2022] DELETE /identity/v3/users/03f4d15861a84b24937edfa81221fc7b => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:21.454533 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00a634-f58e-4e05-adae-3d3c56438451 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:21.455703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00a634-f58e-4e05-adae-3d3c56438451 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:21.456205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00a634-f58e-4e05-adae-3d3c56438451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:21.509729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-db00a634-f58e-4e05-adae-3d3c56438451 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:21.511418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-db00a634-f58e-4e05-adae-3d3c56438451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:21.532313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:21.655470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1117/2217] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:21.681884 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be343c2-f317-4915-85f2-69ff82099a9b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:21.685216 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1be343c2-f317-4915-85f2-69ff82099a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:21.818757 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:21.840328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be343c2-f317-4915-85f2-69ff82099a9b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:21.845035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa69fe-ed96-43f7-baf4-376a2defa5e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:21.848701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa69fe-ed96-43f7-baf4-376a2defa5e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:21.848701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa69fe-ed96-43f7-baf4-376a2defa5e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:21.848701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa69fe-ed96-43f7-baf4-376a2defa5e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:21.878084 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa69fe-ed96-43f7-baf4-376a2defa5e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:21.883984 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa69fe-ed96-43f7-baf4-376a2defa5e4 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:21.908903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1101/2218] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:23.343802 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00a634-f58e-4e05-adae-3d3c56438451 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:23.346768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-db00a634-f58e-4e05-adae-3d3c56438451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:23.373723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00a634-f58e-4e05-adae-3d3c56438451 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:23.380696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e01a91-2c91-4f28-b95f-6400cd0e6edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:23.382353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e01a91-2c91-4f28-b95f-6400cd0e6edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:23.382804 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e01a91-2c91-4f28-b95f-6400cd0e6edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:23.383745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e01a91-2c91-4f28-b95f-6400cd0e6edc admin admin] PATH_INFO: `/v3/projects/30f69128017743eea63a05b325a2e246` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:23.385707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e01a91-2c91-4f28-b95f-6400cd0e6edc admin admin] RBAC: Authorizing `identity:delete_project(project_id=30f69128017743eea63a05b325a2e246)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:23.390146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e01a91-2c91-4f28-b95f-6400cd0e6edc admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:23.497793 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-03e01a91-2c91-4f28-b95f-6400cd0e6edc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30f69128017743eea63a05b325a2e246', 'request_id': 'req-03e01a91-2c91-4f28-b95f-6400cd0e6edc'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:23.506838 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-03e01a91-2c91-4f28-b95f-6400cd0e6edc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30f69128017743eea63a05b325a2e246', 'request_id': 'req-03e01a91-2c91-4f28-b95f-6400cd0e6edc'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:23.522319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-03e01a91-2c91-4f28-b95f-6400cd0e6edc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30f69128017743eea63a05b325a2e246', 'request_id': 'req-03e01a91-2c91-4f28-b95f-6400cd0e6edc'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:23.536274 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-03e01a91-2c91-4f28-b95f-6400cd0e6edc admin admin] The token cache is being invalidate because project 30f69128017743eea63a05b325a2e246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:23.536708 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-03e01a91-2c91-4f28-b95f-6400cd0e6edc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:23.537469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-03e01a91-2c91-4f28-b95f-6400cd0e6edc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:23.554975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1118/2219] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:23 2022] DELETE /identity/v3/projects/30f69128017743eea63a05b325a2e246 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:23.556915 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa69fe-ed96-43f7-baf4-376a2defa5e4 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:23.573685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa69fe-ed96-43f7-baf4-376a2defa5e4 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:23.728627 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50364d8-dd1e-470f-8dcd-7bb590912faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:23.729493 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50364d8-dd1e-470f-8dcd-7bb590912faf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:23.729859 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50364d8-dd1e-470f-8dcd-7bb590912faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:23.730627 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50364d8-dd1e-470f-8dcd-7bb590912faf admin admin] PATH_INFO: `/v3/users/1a4e434eec5b4f1da9ed8dfab927f63d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:23.733441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50364d8-dd1e-470f-8dcd-7bb590912faf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a4e434eec5b4f1da9ed8dfab927f63d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:23.735432 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50364d8-dd1e-470f-8dcd-7bb590912faf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:23.841794 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b50364d8-dd1e-470f-8dcd-7bb590912faf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a4e434eec5b4f1da9ed8dfab927f63d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:23.859665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b50364d8-dd1e-470f-8dcd-7bb590912faf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a4e434eec5b4f1da9ed8dfab927f63d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:23.873934 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b50364d8-dd1e-470f-8dcd-7bb590912faf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a4e434eec5b4f1da9ed8dfab927f63d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:23.874664 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b50364d8-dd1e-470f-8dcd-7bb590912faf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a4e434eec5b4f1da9ed8dfab927f63d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:23.875330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b50364d8-dd1e-470f-8dcd-7bb590912faf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a4e434eec5b4f1da9ed8dfab927f63d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:23.923741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1102/2220] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:38:23 2022] DELETE /identity/v3/users/1a4e434eec5b4f1da9ed8dfab927f63d => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:24.034340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98267093-a1de-42c6-b9d5-77045e0376af None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:24.034805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98267093-a1de-42c6-b9d5-77045e0376af None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:24.035794 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98267093-a1de-42c6-b9d5-77045e0376af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:24.068656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-98267093-a1de-42c6-b9d5-77045e0376af None None] MFA Rules not processed for user `dc246e144afc46698000f7f53ef6489e`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:24.070706 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-98267093-a1de-42c6-b9d5-77045e0376af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:24.169295 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1119/2221] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:24.709342 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50364d8-dd1e-470f-8dcd-7bb590912faf admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:24.710936 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b50364d8-dd1e-470f-8dcd-7bb590912faf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:24.842374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50364d8-dd1e-470f-8dcd-7bb590912faf admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:24.851384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5893f-97e4-403e-8b30-ac35650cf895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:24.852548 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5893f-97e4-403e-8b30-ac35650cf895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:24.852930 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5893f-97e4-403e-8b30-ac35650cf895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:24.853220 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5893f-97e4-403e-8b30-ac35650cf895 admin admin] PATH_INFO: `/v3/projects/f687637bd4164671890f3f90b66fd78c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:24.854971 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5893f-97e4-403e-8b30-ac35650cf895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f687637bd4164671890f3f90b66fd78c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:24.858667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5893f-97e4-403e-8b30-ac35650cf895 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:24.873643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:24.931910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-add5893f-97e4-403e-8b30-ac35650cf895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f687637bd4164671890f3f90b66fd78c', 'request_id': 'req-add5893f-97e4-403e-8b30-ac35650cf895'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:24.945699 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-add5893f-97e4-403e-8b30-ac35650cf895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f687637bd4164671890f3f90b66fd78c', 'request_id': 'req-add5893f-97e4-403e-8b30-ac35650cf895'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:24.955569 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-add5893f-97e4-403e-8b30-ac35650cf895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f687637bd4164671890f3f90b66fd78c', 'request_id': 'req-add5893f-97e4-403e-8b30-ac35650cf895'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:24.977028 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-add5893f-97e4-403e-8b30-ac35650cf895 admin admin] The token cache is being invalidate because project f687637bd4164671890f3f90b66fd78c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:24.978027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-add5893f-97e4-403e-8b30-ac35650cf895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:24.979216 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-add5893f-97e4-403e-8b30-ac35650cf895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:24.985851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1103/2222] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:24 2022] DELETE /identity/v3/projects/f687637bd4164671890f3f90b66fd78c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:24.987756 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98267093-a1de-42c6-b9d5-77045e0376af None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:25.007832 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98267093-a1de-42c6-b9d5-77045e0376af None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:25.127475 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:25.188187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc9beb-d337-43ba-88a8-503240795dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:25.189093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc9beb-d337-43ba-88a8-503240795dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:25.189375 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc9beb-d337-43ba-88a8-503240795dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:25.189662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc9beb-d337-43ba-88a8-503240795dfe admin admin] PATH_INFO: `/v3/users/74782852e2b5408e99e7b26acfbd2d8f` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:25.191040 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc9beb-d337-43ba-88a8-503240795dfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=74782852e2b5408e99e7b26acfbd2d8f)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:25.194053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc9beb-d337-43ba-88a8-503240795dfe admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:25.295990 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-46bc9beb-d337-43ba-88a8-503240795dfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74782852e2b5408e99e7b26acfbd2d8f'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:25.320878 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-46bc9beb-d337-43ba-88a8-503240795dfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74782852e2b5408e99e7b26acfbd2d8f'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:25.330866 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-46bc9beb-d337-43ba-88a8-503240795dfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74782852e2b5408e99e7b26acfbd2d8f'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:25.341080 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-46bc9beb-d337-43ba-88a8-503240795dfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74782852e2b5408e99e7b26acfbd2d8f'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:25.341893 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-46bc9beb-d337-43ba-88a8-503240795dfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74782852e2b5408e99e7b26acfbd2d8f'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:25.348936 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1120/2223] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:38:24 2022] DELETE /identity/v3/users/74782852e2b5408e99e7b26acfbd2d8f => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:25.776909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5893f-97e4-403e-8b30-ac35650cf895 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:25.778639 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-add5893f-97e4-403e-8b30-ac35650cf895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:25.832708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5893f-97e4-403e-8b30-ac35650cf895 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:26.016944 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ec4e9-cc4b-4fcf-9592-d326981d6b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:26.017915 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ec4e9-cc4b-4fcf-9592-d326981d6b7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:26.018209 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ec4e9-cc4b-4fcf-9592-d326981d6b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:26.018550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ec4e9-cc4b-4fcf-9592-d326981d6b7a admin admin] PATH_INFO: `/v3/projects/e8831a73e2e1432eb1c2719b54b3a512` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:26.020755 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ec4e9-cc4b-4fcf-9592-d326981d6b7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8831a73e2e1432eb1c2719b54b3a512)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:26.023479 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ec4e9-cc4b-4fcf-9592-d326981d6b7a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:26.149637 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-438ec4e9-cc4b-4fcf-9592-d326981d6b7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8831a73e2e1432eb1c2719b54b3a512', 'request_id': 'req-438ec4e9-cc4b-4fcf-9592-d326981d6b7a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:26.175622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-438ec4e9-cc4b-4fcf-9592-d326981d6b7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8831a73e2e1432eb1c2719b54b3a512', 'request_id': 'req-438ec4e9-cc4b-4fcf-9592-d326981d6b7a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:26.182719 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-438ec4e9-cc4b-4fcf-9592-d326981d6b7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8831a73e2e1432eb1c2719b54b3a512', 'request_id': 'req-438ec4e9-cc4b-4fcf-9592-d326981d6b7a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:26.216558 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-438ec4e9-cc4b-4fcf-9592-d326981d6b7a admin admin] The token cache is being invalidate because project e8831a73e2e1432eb1c2719b54b3a512 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:26.219685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-438ec4e9-cc4b-4fcf-9592-d326981d6b7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:26.221007 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-438ec4e9-cc4b-4fcf-9592-d326981d6b7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:26.238013 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1104/2224] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:25 2022] DELETE /identity/v3/projects/e8831a73e2e1432eb1c2719b54b3a512 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:27.242667 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd35cda-6eca-4e99-a390-e21e5a0e79a0 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:27.243511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd35cda-6eca-4e99-a390-e21e5a0e79a0 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:27.243511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd35cda-6eca-4e99-a390-e21e5a0e79a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:27.404443 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-0cd35cda-6eca-4e99-a390-e21e5a0e79a0 None None] MFA Rules not processed for user `f523b529906f4fa989d0e836d6a08944`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:27.407448 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-0cd35cda-6eca-4e99-a390-e21e5a0e79a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:27.630800 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1121/2225] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:27.633007 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ec4e9-cc4b-4fcf-9592-d326981d6b7a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:27.634774 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-438ec4e9-cc4b-4fcf-9592-d326981d6b7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:27.692193 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:27.837937 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ec4e9-cc4b-4fcf-9592-d326981d6b7a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:27.845110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d7078-6651-4348-a5cf-7b5c9ddf65c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:27.846035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d7078-6651-4348-a5cf-7b5c9ddf65c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:27.846427 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d7078-6651-4348-a5cf-7b5c9ddf65c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:27.846847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d7078-6651-4348-a5cf-7b5c9ddf65c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:27.848041 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d7078-6651-4348-a5cf-7b5c9ddf65c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:27.849692 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d7078-6651-4348-a5cf-7b5c9ddf65c5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:27.898414 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1105/2226] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:38:27 2022] POST /identity/v3/projects => generated 379 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:27.900259 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd35cda-6eca-4e99-a390-e21e5a0e79a0 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:27.946245 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd35cda-6eca-4e99-a390-e21e5a0e79a0 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:28.024871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:28.086454 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a60429-e86a-4da8-a183-7d8d70e757fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:28.090480 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a60429-e86a-4da8-a183-7d8d70e757fe admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:28.092133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a60429-e86a-4da8-a183-7d8d70e757fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:28.092778 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a60429-e86a-4da8-a183-7d8d70e757fe admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:28.094605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a60429-e86a-4da8-a183-7d8d70e757fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:28.102620 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a60429-e86a-4da8-a183-7d8d70e757fe admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:28.147713 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1122/2227] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:38:27 2022] POST /identity/v3/users => generated 296 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:28.151976 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d7078-6651-4348-a5cf-7b5c9ddf65c5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:28.180126 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d7078-6651-4348-a5cf-7b5c9ddf65c5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:28.194678 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950f6f5-5d18-4af7-9926-63747eaa5d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:28.194678 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950f6f5-5d18-4af7-9926-63747eaa5d58 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:28.194678 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950f6f5-5d18-4af7-9926-63747eaa5d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:28.194678 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950f6f5-5d18-4af7-9926-63747eaa5d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:28.195197 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950f6f5-5d18-4af7-9926-63747eaa5d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:28.195197 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950f6f5-5d18-4af7-9926-63747eaa5d58 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:28.209294 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1106/2228] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:28.212846 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a60429-e86a-4da8-a183-7d8d70e757fe admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:28.229846 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a60429-e86a-4da8-a183-7d8d70e757fe admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:28.249770 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f3187-2dba-4a73-b485-344cdcd5ad78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:28.251807 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f3187-2dba-4a73-b485-344cdcd5ad78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:28.252826 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f3187-2dba-4a73-b485-344cdcd5ad78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:28.253366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f3187-2dba-4a73-b485-344cdcd5ad78 admin admin] PATH_INFO: `/v3/projects/d6d7cdd3e4cc418e9d1bb2280f92ae6d/users/cc33294baa794bc3b67fb627e42bf3fc/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:28.276213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f3187-2dba-4a73-b485-344cdcd5ad78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6d7cdd3e4cc418e9d1bb2280f92ae6d, user_id=cc33294baa794bc3b67fb627e42bf3fc, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:28.279536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f3187-2dba-4a73-b485-344cdcd5ad78 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:28.346795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1123/2229] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:28 2022] PUT /identity/v3/projects/d6d7cdd3e4cc418e9d1bb2280f92ae6d/users/cc33294baa794bc3b67fb627e42bf3fc/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:28.348229 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950f6f5-5d18-4af7-9926-63747eaa5d58 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:28.384776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950f6f5-5d18-4af7-9926-63747eaa5d58 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:28.577870 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb68b84-6812-42d4-99b4-83347ed233b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:28.579923 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb68b84-6812-42d4-99b4-83347ed233b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:28.580384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb68b84-6812-42d4-99b4-83347ed233b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:28.581607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb68b84-6812-42d4-99b4-83347ed233b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:28.582474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb68b84-6812-42d4-99b4-83347ed233b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:28.585074 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb68b84-6812-42d4-99b4-83347ed233b6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:28.644870 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1107/2230] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:28.646076 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f3187-2dba-4a73-b485-344cdcd5ad78 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:28.662362 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f3187-2dba-4a73-b485-344cdcd5ad78 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:28.681254 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea71a5-8acf-49a0-94b1-90d726d8f88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:28.687660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea71a5-8acf-49a0-94b1-90d726d8f88e admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:28.688306 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea71a5-8acf-49a0-94b1-90d726d8f88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:28.689579 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea71a5-8acf-49a0-94b1-90d726d8f88e admin admin] PATH_INFO: `/v3/projects/d6d7cdd3e4cc418e9d1bb2280f92ae6d/users/cc33294baa794bc3b67fb627e42bf3fc/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:28.692915 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea71a5-8acf-49a0-94b1-90d726d8f88e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6d7cdd3e4cc418e9d1bb2280f92ae6d, user_id=cc33294baa794bc3b67fb627e42bf3fc, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:28.705007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea71a5-8acf-49a0-94b1-90d726d8f88e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:28.764280 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1124/2231] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:28 2022] PUT /identity/v3/projects/d6d7cdd3e4cc418e9d1bb2280f92ae6d/users/cc33294baa794bc3b67fb627e42bf3fc/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:38:32.211395 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4811b-df11-4c34-a4f5-b7ac0c51e16b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:32.212316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4811b-df11-4c34-a4f5-b7ac0c51e16b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:32.212755 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4811b-df11-4c34-a4f5-b7ac0c51e16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:32.240600 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-35c4811b-df11-4c34-a4f5-b7ac0c51e16b None None] MFA Rules not processed for user `03486c7630314392b85271455d12aba2`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:32.242307 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-35c4811b-df11-4c34-a4f5-b7ac0c51e16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:32.300201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:32.327026 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1108/2232] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:38.662026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049070f-b68c-4fa4-a456-68a5d43c833b None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:38.663203 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049070f-b68c-4fa4-a456-68a5d43c833b None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:38.663822 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049070f-b68c-4fa4-a456-68a5d43c833b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:38.763234 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-d049070f-b68c-4fa4-a456-68a5d43c833b None None] MFA Rules not processed for user `cc33294baa794bc3b67fb627e42bf3fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:38.766645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d049070f-b68c-4fa4-a456-68a5d43c833b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:39.012255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1125/2233] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:39.026842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4811b-df11-4c34-a4f5-b7ac0c51e16b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:39.029330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-35c4811b-df11-4c34-a4f5-b7ac0c51e16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:39.170304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4811b-df11-4c34-a4f5-b7ac0c51e16b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:39.176690 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426a1e0-bee3-411c-bfcd-e23c81fdd50e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:39.177567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426a1e0-bee3-411c-bfcd-e23c81fdd50e service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:39.178005 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426a1e0-bee3-411c-bfcd-e23c81fdd50e service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:39.178458 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426a1e0-bee3-411c-bfcd-e23c81fdd50e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:39.278131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426a1e0-bee3-411c-bfcd-e23c81fdd50e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:39.280096 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426a1e0-bee3-411c-bfcd-e23c81fdd50e service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:39.302851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1109/2234] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:43.574608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52f23c-96fb-4f58-937f-6031eb8ed57f None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:43.575594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52f23c-96fb-4f58-937f-6031eb8ed57f None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:43.576508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52f23c-96fb-4f58-937f-6031eb8ed57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:43.611962 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-6f52f23c-96fb-4f58-937f-6031eb8ed57f None None] MFA Rules not processed for user `df1aeb2be41c49819b811ecfa47d4155`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:43.613844 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-6f52f23c-96fb-4f58-937f-6031eb8ed57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:43.745066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1126/2235] 10.222.0.43 () {58 vars in 967 bytes} [Thu Jun 23 22:38:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:43.771254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426a1e0-bee3-411c-bfcd-e23c81fdd50e service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:43.773615 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4426a1e0-bee3-411c-bfcd-e23c81fdd50e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:43.881200 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426a1e0-bee3-411c-bfcd-e23c81fdd50e service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:43.887231 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47ff52-e2c6-4ccc-99ee-722d3885423a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ce35c43e8294f4cbc880f2219a6eb76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:43.888460 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47ff52-e2c6-4ccc-99ee-722d3885423a service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:43.888915 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47ff52-e2c6-4ccc-99ee-722d3885423a service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:43.889346 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47ff52-e2c6-4ccc-99ee-722d3885423a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:43.900135 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47ff52-e2c6-4ccc-99ee-722d3885423a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:43.902708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b47ff52-e2c6-4ccc-99ee-722d3885423a service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:43.914945 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:43.935719 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1110/2236] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 22:38:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:38:44.472644 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52f23c-96fb-4f58-937f-6031eb8ed57f None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:44.475140 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-6f52f23c-96fb-4f58-937f-6031eb8ed57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:44.483401 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:44.682518 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52f23c-96fb-4f58-937f-6031eb8ed57f None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:44.700138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6194cb-21ca-407a-beb0-04cfb42dd81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:44.701161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6194cb-21ca-407a-beb0-04cfb42dd81e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:44.701445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6194cb-21ca-407a-beb0-04cfb42dd81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:44.701724 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6194cb-21ca-407a-beb0-04cfb42dd81e admin admin] PATH_INFO: `/v3/users/25569f5a20244d9380aaed41b7278e19` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:44.703939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6194cb-21ca-407a-beb0-04cfb42dd81e admin admin] RBAC: Authorizing `identity:delete_user(user_id=25569f5a20244d9380aaed41b7278e19)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:44.705956 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6194cb-21ca-407a-beb0-04cfb42dd81e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:44.845844 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-af6194cb-21ca-407a-beb0-04cfb42dd81e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25569f5a20244d9380aaed41b7278e19'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:44.870374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-af6194cb-21ca-407a-beb0-04cfb42dd81e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25569f5a20244d9380aaed41b7278e19'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:44.883272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-af6194cb-21ca-407a-beb0-04cfb42dd81e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25569f5a20244d9380aaed41b7278e19'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:44.896698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-af6194cb-21ca-407a-beb0-04cfb42dd81e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25569f5a20244d9380aaed41b7278e19'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:44.900818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-af6194cb-21ca-407a-beb0-04cfb42dd81e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25569f5a20244d9380aaed41b7278e19'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:44.913152 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1127/2237] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:38:44 2022] DELETE /identity/v3/users/25569f5a20244d9380aaed41b7278e19 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:44.921679 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c4fc3-5c09-4025-837b-5c02792ab9d8 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:44.922499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c4fc3-5c09-4025-837b-5c02792ab9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:44.922953 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c4fc3-5c09-4025-837b-5c02792ab9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:44.958185 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e19c4fc3-5c09-4025-837b-5c02792ab9d8 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:44.961297 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e19c4fc3-5c09-4025-837b-5c02792ab9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:45.076590 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1111/2238] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:45.092369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6194cb-21ca-407a-beb0-04cfb42dd81e admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:45.094347 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-af6194cb-21ca-407a-beb0-04cfb42dd81e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:45.179401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cda454-24bb-4662-b0b3-39ce48580f8c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:45.179790 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cda454-24bb-4662-b0b3-39ce48580f8c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:45.180152 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cda454-24bb-4662-b0b3-39ce48580f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:45.203926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-62cda454-24bb-4662-b0b3-39ce48580f8c None None] MFA Rules not processed for user `df1aeb2be41c49819b811ecfa47d4155`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:45.206239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-62cda454-24bb-4662-b0b3-39ce48580f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:45.233339 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6194cb-21ca-407a-beb0-04cfb42dd81e admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:45.239430 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8a3e7-9e78-4e20-9365-583dd54ca6e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:45.240017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8a3e7-9e78-4e20-9365-583dd54ca6e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:45.240468 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8a3e7-9e78-4e20-9365-583dd54ca6e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:45.240933 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8a3e7-9e78-4e20-9365-583dd54ca6e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:45.257354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8a3e7-9e78-4e20-9365-583dd54ca6e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:45.262968 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8a3e7-9e78-4e20-9365-583dd54ca6e8 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:45.281419 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1112/2239] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 22:38:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:38:45.291058 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1128/2240] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:45.298026 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cda454-24bb-4662-b0b3-39ce48580f8c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:45.299425 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-62cda454-24bb-4662-b0b3-39ce48580f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:45.412977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cda454-24bb-4662-b0b3-39ce48580f8c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:45.427746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbec38-6f0e-4a22-a4ed-48abe1ebdfcf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ce35c43e8294f4cbc880f2219a6eb76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:45.429506 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbec38-6f0e-4a22-a4ed-48abe1ebdfcf service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:45.430060 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbec38-6f0e-4a22-a4ed-48abe1ebdfcf service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:45.430729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbec38-6f0e-4a22-a4ed-48abe1ebdfcf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:45.443579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbec38-6f0e-4a22-a4ed-48abe1ebdfcf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:45.445471 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbec38-6f0e-4a22-a4ed-48abe1ebdfcf service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:45.456934 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1113/2241] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 22:38:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:38:45.754293 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8a3e7-9e78-4e20-9365-583dd54ca6e8 service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:45.756824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-6ed8a3e7-9e78-4e20-9365-583dd54ca6e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:45.780743 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8a3e7-9e78-4e20-9365-583dd54ca6e8 service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:45.801024 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ad8f1-5660-4f3e-84cd-6acc2d4347c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:45.802467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ad8f1-5660-4f3e-84cd-6acc2d4347c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:45.802910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ad8f1-5660-4f3e-84cd-6acc2d4347c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:45.803384 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ad8f1-5660-4f3e-84cd-6acc2d4347c2 admin admin] PATH_INFO: `/v3/projects/527d0b7e41ec40b99c9b2f383a0baa73` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:45.805522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ad8f1-5660-4f3e-84cd-6acc2d4347c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=527d0b7e41ec40b99c9b2f383a0baa73)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:45.808819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ad8f1-5660-4f3e-84cd-6acc2d4347c2 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:45.812525 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:45.998489 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c76ad8f1-5660-4f3e-84cd-6acc2d4347c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '527d0b7e41ec40b99c9b2f383a0baa73', 'request_id': 'req-c76ad8f1-5660-4f3e-84cd-6acc2d4347c2'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:46.012353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c76ad8f1-5660-4f3e-84cd-6acc2d4347c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '527d0b7e41ec40b99c9b2f383a0baa73', 'request_id': 'req-c76ad8f1-5660-4f3e-84cd-6acc2d4347c2'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:46.029598 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c76ad8f1-5660-4f3e-84cd-6acc2d4347c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '527d0b7e41ec40b99c9b2f383a0baa73', 'request_id': 'req-c76ad8f1-5660-4f3e-84cd-6acc2d4347c2'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:46.054181 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c76ad8f1-5660-4f3e-84cd-6acc2d4347c2 admin admin] The token cache is being invalidate because project 527d0b7e41ec40b99c9b2f383a0baa73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:46.054297 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c76ad8f1-5660-4f3e-84cd-6acc2d4347c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:46.054976 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c76ad8f1-5660-4f3e-84cd-6acc2d4347c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:46.057299 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1129/2242] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:45 2022] DELETE /identity/v3/projects/527d0b7e41ec40b99c9b2f383a0baa73 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:46.061725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbec38-6f0e-4a22-a4ed-48abe1ebdfcf service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:46.098020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbec38-6f0e-4a22-a4ed-48abe1ebdfcf service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:46.208461 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:46.288851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82697030-8f11-4e18-b6a5-aea2b4a67eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:46.295381 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82697030-8f11-4e18-b6a5-aea2b4a67eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:46.296032 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82697030-8f11-4e18-b6a5-aea2b4a67eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:46.296914 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82697030-8f11-4e18-b6a5-aea2b4a67eef admin admin] PATH_INFO: `/v3/users/24704fe11803499e852ecdd888977856` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:46.298762 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82697030-8f11-4e18-b6a5-aea2b4a67eef admin admin] RBAC: Authorizing `identity:delete_user(user_id=24704fe11803499e852ecdd888977856)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:46.300986 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82697030-8f11-4e18-b6a5-aea2b4a67eef admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:46.409388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-82697030-8f11-4e18-b6a5-aea2b4a67eef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24704fe11803499e852ecdd888977856'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:46.420372 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-82697030-8f11-4e18-b6a5-aea2b4a67eef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24704fe11803499e852ecdd888977856'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:46.435676 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-82697030-8f11-4e18-b6a5-aea2b4a67eef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24704fe11803499e852ecdd888977856'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:46.436494 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-82697030-8f11-4e18-b6a5-aea2b4a67eef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24704fe11803499e852ecdd888977856'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:46.437255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-82697030-8f11-4e18-b6a5-aea2b4a67eef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24704fe11803499e852ecdd888977856'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:46.453991 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39816614-3780-4964-b07e-14669aeffafc None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:46.454288 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39816614-3780-4964-b07e-14669aeffafc None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:46.454891 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39816614-3780-4964-b07e-14669aeffafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:46.472754 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1114/2243] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:38:46 2022] DELETE /identity/v3/users/24704fe11803499e852ecdd888977856 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:46.488506 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-39816614-3780-4964-b07e-14669aeffafc None None] MFA Rules not processed for user `df1aeb2be41c49819b811ecfa47d4155`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:46.489789 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-39816614-3780-4964-b07e-14669aeffafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:46.568868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1130/2244] 10.222.0.43 () {58 vars in 967 bytes} [Thu Jun 23 22:38:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:46.591560 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82697030-8f11-4e18-b6a5-aea2b4a67eef admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:46.593408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-82697030-8f11-4e18-b6a5-aea2b4a67eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:46.723749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82697030-8f11-4e18-b6a5-aea2b4a67eef admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:46.735543 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcfd37-af78-49fa-88aa-1dd449c74cc8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ce35c43e8294f4cbc880f2219a6eb76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:46.736507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcfd37-af78-49fa-88aa-1dd449c74cc8 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:46.736999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcfd37-af78-49fa-88aa-1dd449c74cc8 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:46.737482 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcfd37-af78-49fa-88aa-1dd449c74cc8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:46.751268 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcfd37-af78-49fa-88aa-1dd449c74cc8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:46.753514 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcfd37-af78-49fa-88aa-1dd449c74cc8 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:46.789294 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1115/2245] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 22:38:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:38:47.118490 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc165ef5-17b7-4f80-b9a8-83be0ac3f675 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:47.118984 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc165ef5-17b7-4f80-b9a8-83be0ac3f675 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:47.119273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc165ef5-17b7-4f80-b9a8-83be0ac3f675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:47.147795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-cc165ef5-17b7-4f80-b9a8-83be0ac3f675 None None] MFA Rules not processed for user `df1aeb2be41c49819b811ecfa47d4155`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:47.149293 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-cc165ef5-17b7-4f80-b9a8-83be0ac3f675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:47.165508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1131/2246] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 22:38:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:47.177866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcfd37-af78-49fa-88aa-1dd449c74cc8 service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:47.191661 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcfd37-af78-49fa-88aa-1dd449c74cc8 service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:47.197743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af893f-85fc-4e8a-8476-e1fa2946e2df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ce35c43e8294f4cbc880f2219a6eb76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:47.198388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af893f-85fc-4e8a-8476-e1fa2946e2df service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:47.199669 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af893f-85fc-4e8a-8476-e1fa2946e2df service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:47.200145 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af893f-85fc-4e8a-8476-e1fa2946e2df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:47.212957 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af893f-85fc-4e8a-8476-e1fa2946e2df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:47.221113 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af893f-85fc-4e8a-8476-e1fa2946e2df service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:47.232361 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1116/2247] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 22:38:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 22:38:47.236299 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc165ef5-17b7-4f80-b9a8-83be0ac3f675 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:47.238302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-cc165ef5-17b7-4f80-b9a8-83be0ac3f675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:47.330745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc165ef5-17b7-4f80-b9a8-83be0ac3f675 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:47.341896 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af893f-85fc-4e8a-8476-e1fa2946e2df service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:47.345325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-36af893f-85fc-4e8a-8476-e1fa2946e2df service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:47.346139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972579f3-bb23-472d-bf27-bb668865a2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:47.347330 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972579f3-bb23-472d-bf27-bb668865a2dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:47.347537 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972579f3-bb23-472d-bf27-bb668865a2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:47.347895 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972579f3-bb23-472d-bf27-bb668865a2dd admin admin] PATH_INFO: `/v3/users/bbe630f5c7c243588c5bd33023775d69` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:47.350582 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972579f3-bb23-472d-bf27-bb668865a2dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbe630f5c7c243588c5bd33023775d69)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:47.352707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972579f3-bb23-472d-bf27-bb668865a2dd admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:47.367398 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af893f-85fc-4e8a-8476-e1fa2946e2df service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:47.387410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72f748-e635-4cdc-bb89-4b61fac4a816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:47.388370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72f748-e635-4cdc-bb89-4b61fac4a816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:47.388824 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72f748-e635-4cdc-bb89-4b61fac4a816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:47.389276 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72f748-e635-4cdc-bb89-4b61fac4a816 admin admin] PATH_INFO: `/v3/projects/0e158e53194f4953a47067a045c3aa9b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:47.390958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72f748-e635-4cdc-bb89-4b61fac4a816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e158e53194f4953a47067a045c3aa9b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:47.393257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72f748-e635-4cdc-bb89-4b61fac4a816 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:47.403881 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:47.422060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-972579f3-bb23-472d-bf27-bb668865a2dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbe630f5c7c243588c5bd33023775d69'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:47.422780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:47.439816 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-972579f3-bb23-472d-bf27-bb668865a2dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbe630f5c7c243588c5bd33023775d69'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:47.449711 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-972579f3-bb23-472d-bf27-bb668865a2dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbe630f5c7c243588c5bd33023775d69'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:47.458592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1f72f748-e635-4cdc-bb89-4b61fac4a816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e158e53194f4953a47067a045c3aa9b', 'request_id': 'req-1f72f748-e635-4cdc-bb89-4b61fac4a816'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:47.459614 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-972579f3-bb23-472d-bf27-bb668865a2dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbe630f5c7c243588c5bd33023775d69'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:47.461766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-972579f3-bb23-472d-bf27-bb668865a2dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbe630f5c7c243588c5bd33023775d69'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:47.467460 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1f72f748-e635-4cdc-bb89-4b61fac4a816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e158e53194f4953a47067a045c3aa9b', 'request_id': 'req-1f72f748-e635-4cdc-bb89-4b61fac4a816'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:47.475103 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1132/2248] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:38:47 2022] DELETE /identity/v3/users/bbe630f5c7c243588c5bd33023775d69 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:47.484700 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1f72f748-e635-4cdc-bb89-4b61fac4a816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e158e53194f4953a47067a045c3aa9b', 'request_id': 'req-1f72f748-e635-4cdc-bb89-4b61fac4a816'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:47.519943 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1f72f748-e635-4cdc-bb89-4b61fac4a816 admin admin] The token cache is being invalidate because project 0e158e53194f4953a47067a045c3aa9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:47.520267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1f72f748-e635-4cdc-bb89-4b61fac4a816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:47.522025 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-1f72f748-e635-4cdc-bb89-4b61fac4a816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:47.524066 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1117/2249] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:47 2022] DELETE /identity/v3/projects/0e158e53194f4953a47067a045c3aa9b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:47.533290 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0797e75-6112-42e1-84cb-4dbcd7f30ebf None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:47.533615 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0797e75-6112-42e1-84cb-4dbcd7f30ebf None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:47.533924 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0797e75-6112-42e1-84cb-4dbcd7f30ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:47.604917 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-b0797e75-6112-42e1-84cb-4dbcd7f30ebf None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:47.606960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b0797e75-6112-42e1-84cb-4dbcd7f30ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:47.664718 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415b555-8fde-47ee-95f8-9323431ab81a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:47.665087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415b555-8fde-47ee-95f8-9323431ab81a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:47.665525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415b555-8fde-47ee-95f8-9323431ab81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:47.687098 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1133/2250] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:47.696761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd7738-0c7c-4d0e-b9f6-153a8e4d1c81 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:47.696946 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd7738-0c7c-4d0e-b9f6-153a8e4d1c81 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:47.697283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd7738-0c7c-4d0e-b9f6-153a8e4d1c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:47.709998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7415b555-8fde-47ee-95f8-9323431ab81a None None] MFA Rules not processed for user `df1aeb2be41c49819b811ecfa47d4155`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:47.716187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7415b555-8fde-47ee-95f8-9323431ab81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:47.730136 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-1bfd7738-0c7c-4d0e-b9f6-153a8e4d1c81 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:47.732872 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1bfd7738-0c7c-4d0e-b9f6-153a8e4d1c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:47.744974 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1134/2251] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:47.748329 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd7738-0c7c-4d0e-b9f6-153a8e4d1c81 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:47.748329 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1bfd7738-0c7c-4d0e-b9f6-153a8e4d1c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:47.761193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd7738-0c7c-4d0e-b9f6-153a8e4d1c81 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:47.767312 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af9768-57c9-4f11-be99-8400ae96070b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:47.767806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af9768-57c9-4f11-be99-8400ae96070b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:47.768601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af9768-57c9-4f11-be99-8400ae96070b admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:47.768986 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af9768-57c9-4f11-be99-8400ae96070b admin admin] PATH_INFO: `/v3/projects/e4fb6c541ee04ebea81001c24c67d779` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:47.773257 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af9768-57c9-4f11-be99-8400ae96070b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4fb6c541ee04ebea81001c24c67d779)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:47.776094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af9768-57c9-4f11-be99-8400ae96070b admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:47.819348 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1118/2252] 10.222.0.43 () {58 vars in 967 bytes} [Thu Jun 23 22:38:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:38:47.820233 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415b555-8fde-47ee-95f8-9323431ab81a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:47.842148 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415b555-8fde-47ee-95f8-9323431ab81a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:47.854128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b0af9768-57c9-4f11-be99-8400ae96070b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4fb6c541ee04ebea81001c24c67d779', 'request_id': 'req-b0af9768-57c9-4f11-be99-8400ae96070b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:47.861930 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b0af9768-57c9-4f11-be99-8400ae96070b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4fb6c541ee04ebea81001c24c67d779', 'request_id': 'req-b0af9768-57c9-4f11-be99-8400ae96070b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:47.872540 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b0af9768-57c9-4f11-be99-8400ae96070b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4fb6c541ee04ebea81001c24c67d779', 'request_id': 'req-b0af9768-57c9-4f11-be99-8400ae96070b'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:47.881811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:47.887496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b0af9768-57c9-4f11-be99-8400ae96070b admin admin] The token cache is being invalidate because project e4fb6c541ee04ebea81001c24c67d779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:47.888145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b0af9768-57c9-4f11-be99-8400ae96070b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:47.890783 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b0af9768-57c9-4f11-be99-8400ae96070b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:47.894129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1135/2253] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:47 2022] DELETE /identity/v3/projects/e4fb6c541ee04ebea81001c24c67d779 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:47.895176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af9768-57c9-4f11-be99-8400ae96070b admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:47.896775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b0af9768-57c9-4f11-be99-8400ae96070b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:47.923355 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d7c94-708a-4dd8-b636-29654340b24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:47.925116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d7c94-708a-4dd8-b636-29654340b24d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:47.927201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d7c94-708a-4dd8-b636-29654340b24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:47.927562 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d7c94-708a-4dd8-b636-29654340b24d admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:47.928344 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d7c94-708a-4dd8-b636-29654340b24d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:38:47.929480 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d7c94-708a-4dd8-b636-29654340b24d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:47.932654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d7c94-708a-4dd8-b636-29654340b24d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:47.954088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1119/2254] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:38:47.955308 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec7e88-d9e8-484f-b5a3-81eb52d743b5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:47.955606 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec7e88-d9e8-484f-b5a3-81eb52d743b5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:47.955892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec7e88-d9e8-484f-b5a3-81eb52d743b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:47.976275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-14ec7e88-d9e8-484f-b5a3-81eb52d743b5 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:47.977712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-14ec7e88-d9e8-484f-b5a3-81eb52d743b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:47.988894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1120/2255] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:47.989697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec7e88-d9e8-484f-b5a3-81eb52d743b5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:47.990973 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-14ec7e88-d9e8-484f-b5a3-81eb52d743b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:48.001570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af9768-57c9-4f11-be99-8400ae96070b admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:48.003326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec7e88-d9e8-484f-b5a3-81eb52d743b5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:48.006729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e963b7c-1f1a-4b64-a1b6-2328e1044a15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ce35c43e8294f4cbc880f2219a6eb76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:48.007477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e963b7c-1f1a-4b64-a1b6-2328e1044a15 service swift] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.007826 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e963b7c-1f1a-4b64-a1b6-2328e1044a15 service swift] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.008011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc65d3-497a-4e8e-b49e-f4fbc381532d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:48.008173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e963b7c-1f1a-4b64-a1b6-2328e1044a15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.008664 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc65d3-497a-4e8e-b49e-f4fbc381532d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.009060 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc65d3-497a-4e8e-b49e-f4fbc381532d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.009345 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc65d3-497a-4e8e-b49e-f4fbc381532d admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.010011 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8e963b7c-1f1a-4b64-a1b6-2328e1044a15 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:48.010204 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc65d3-497a-4e8e-b49e-f4fbc381532d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:48.013130 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc65d3-497a-4e8e-b49e-f4fbc381532d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:48.044254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1121/2256] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:38:47 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:48.044254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa6387-aded-4369-9282-10285b8e93ef None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.044254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa6387-aded-4369-9282-10285b8e93ef None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.044254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa6387-aded-4369-9282-10285b8e93ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.064568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-86fa6387-aded-4369-9282-10285b8e93ef None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:48.065792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-86fa6387-aded-4369-9282-10285b8e93ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:48.081032 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:48.083158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e963b7c-1f1a-4b64-a1b6-2328e1044a15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:48.084975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e963b7c-1f1a-4b64-a1b6-2328e1044a15 service swift] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:48.096447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1136/2257] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 22:38:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:38:48.097512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e963b7c-1f1a-4b64-a1b6-2328e1044a15 service swift] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:48.110124 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e963b7c-1f1a-4b64-a1b6-2328e1044a15 service swift] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:48.165050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1122/2258] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:48.172021 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa6387-aded-4369-9282-10285b8e93ef None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:48.184047 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa6387-aded-4369-9282-10285b8e93ef None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:48.189849 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a69053-52a5-4e24-a98d-87ed32e638ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:48.190817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a69053-52a5-4e24-a98d-87ed32e638ad admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.192227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a69053-52a5-4e24-a98d-87ed32e638ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.192587 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a69053-52a5-4e24-a98d-87ed32e638ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.193621 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a69053-52a5-4e24-a98d-87ed32e638ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:38:48.193778 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a69053-52a5-4e24-a98d-87ed32e638ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:48.198355 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a69053-52a5-4e24-a98d-87ed32e638ad admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:48.209648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef49a1-8c02-4e9a-becb-d4f282818890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:48.210473 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef49a1-8c02-4e9a-becb-d4f282818890 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.210764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef49a1-8c02-4e9a-becb-d4f282818890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.211105 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef49a1-8c02-4e9a-becb-d4f282818890 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.211811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef49a1-8c02-4e9a-becb-d4f282818890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:48.213736 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef49a1-8c02-4e9a-becb-d4f282818890 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:48.214261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1123/2259] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:38:48.219848 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a69053-52a5-4e24-a98d-87ed32e638ad admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:48.241914 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a69053-52a5-4e24-a98d-87ed32e638ad admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:48.245367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1137/2260] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:38:48 2022] POST /identity/v3/users => generated 294 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:48.250943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef49a1-8c02-4e9a-becb-d4f282818890 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:48.257902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7247b7-605a-433f-aae0-e1e9ab733bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:48.262431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7247b7-605a-433f-aae0-e1e9ab733bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.262431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7247b7-605a-433f-aae0-e1e9ab733bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.262431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7247b7-605a-433f-aae0-e1e9ab733bb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.262431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7247b7-605a-433f-aae0-e1e9ab733bb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:48.262706 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7247b7-605a-433f-aae0-e1e9ab733bb7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:48.266485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef49a1-8c02-4e9a-becb-d4f282818890 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:48.272213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e448791-d67f-4091-adec-3c401de73d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:48.273199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e448791-d67f-4091-adec-3c401de73d56 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.273508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e448791-d67f-4091-adec-3c401de73d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.273861 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e448791-d67f-4091-adec-3c401de73d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.274673 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e448791-d67f-4091-adec-3c401de73d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:48.276661 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e448791-d67f-4091-adec-3c401de73d56 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:48.291511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1138/2261] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:48.296480 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e448791-d67f-4091-adec-3c401de73d56 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:48.299241 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1124/2262] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:38:48 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:38:48.303151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7247b7-605a-433f-aae0-e1e9ab733bb7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:48.312917 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e448791-d67f-4091-adec-3c401de73d56 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:48.351427 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:48.418823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e7c2c-d7b7-4b94-ac50-8e6ad47cf92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:48.419877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e7c2c-d7b7-4b94-ac50-8e6ad47cf92f admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.420333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e7c2c-d7b7-4b94-ac50-8e6ad47cf92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.420762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e7c2c-d7b7-4b94-ac50-8e6ad47cf92f admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.421646 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e7c2c-d7b7-4b94-ac50-8e6ad47cf92f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:48.423617 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e7c2c-d7b7-4b94-ac50-8e6ad47cf92f admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:48.430376 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7247b7-605a-433f-aae0-e1e9ab733bb7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:48.436516 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d6772b-68ef-4103-9398-dbe07b156cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:48.437215 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6772b-68ef-4103-9398-dbe07b156cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.437509 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6772b-68ef-4103-9398-dbe07b156cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.437797 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6772b-68ef-4103-9398-dbe07b156cc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.438501 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d6772b-68ef-4103-9398-dbe07b156cc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:48.443475 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d6772b-68ef-4103-9398-dbe07b156cc8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:48.444914 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1139/2263] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:38:48.449758 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e7c2c-d7b7-4b94-ac50-8e6ad47cf92f admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:48.467135 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e7c2c-d7b7-4b94-ac50-8e6ad47cf92f admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:48.473781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad79239-ef56-420f-96f0-def14141d658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:48.475254 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad79239-ef56-420f-96f0-def14141d658 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.478562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad79239-ef56-420f-96f0-def14141d658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.478861 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad79239-ef56-420f-96f0-def14141d658 admin admin] PATH_INFO: `/v3/projects/330603d973f347229d00af15d8357040/users/7faddb9f17084b0186777e17d610628a/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.482685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1125/2264] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:38:48 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:38:48.489858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d6772b-68ef-4103-9398-dbe07b156cc8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:48.513337 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d6772b-68ef-4103-9398-dbe07b156cc8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:48.515328 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad79239-ef56-420f-96f0-def14141d658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=330603d973f347229d00af15d8357040, user_id=7faddb9f17084b0186777e17d610628a, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:48.518310 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad79239-ef56-420f-96f0-def14141d658 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:48.519818 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dba543-20ac-40fc-be65-0db678d8c9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:48.521205 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dba543-20ac-40fc-be65-0db678d8c9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.521808 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dba543-20ac-40fc-be65-0db678d8c9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.522439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dba543-20ac-40fc-be65-0db678d8c9f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.524416 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dba543-20ac-40fc-be65-0db678d8c9f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:48.525682 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dba543-20ac-40fc-be65-0db678d8c9f0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:48.541514 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1140/2265] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:48 2022] PUT /identity/v3/projects/330603d973f347229d00af15d8357040/users/7faddb9f17084b0186777e17d610628a/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:48.548538 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1126/2266] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:48.552559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad79239-ef56-420f-96f0-def14141d658 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:48.555682 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7b65d-113b-4964-8e37-c27599a8c956 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.556136 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7b65d-113b-4964-8e37-c27599a8c956 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.556798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7b65d-113b-4964-8e37-c27599a8c956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.595748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-22b7b65d-113b-4964-8e37-c27599a8c956 None None] MFA Rules not processed for user `7faddb9f17084b0186777e17d610628a`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:48.609139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-22b7b65d-113b-4964-8e37-c27599a8c956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:48.660840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad79239-ef56-420f-96f0-def14141d658 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:48.666491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e48671-607d-4653-9925-ad4d872ce046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:48.672171 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e48671-607d-4653-9925-ad4d872ce046 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.672476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e48671-607d-4653-9925-ad4d872ce046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.672754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e48671-607d-4653-9925-ad4d872ce046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.673939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e48671-607d-4653-9925-ad4d872ce046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:48.675658 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e48671-607d-4653-9925-ad4d872ce046 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:48.701535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1141/2267] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:38:48.701887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:48.703765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e48671-607d-4653-9925-ad4d872ce046 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:48.717413 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e48671-607d-4653-9925-ad4d872ce046 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:48.737796 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90412285-1fc4-430c-bc5b-29e62bde2a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:48.737796 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90412285-1fc4-430c-bc5b-29e62bde2a91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.737796 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90412285-1fc4-430c-bc5b-29e62bde2a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.737796 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90412285-1fc4-430c-bc5b-29e62bde2a91 admin admin] PATH_INFO: `/v3/projects/967d387cb4e24455bfdcea3f32d27755/users/90101c531f0d4e938d16a96da8191c24/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.756859 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90412285-1fc4-430c-bc5b-29e62bde2a91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=967d387cb4e24455bfdcea3f32d27755, user_id=90101c531f0d4e938d16a96da8191c24, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:48.760485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90412285-1fc4-430c-bc5b-29e62bde2a91 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:48.767600 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1127/2268] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:48.775432 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7b65d-113b-4964-8e37-c27599a8c956 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:48.791616 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1142/2269] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:48 2022] PUT /identity/v3/projects/967d387cb4e24455bfdcea3f32d27755/users/90101c531f0d4e938d16a96da8191c24/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:48.793633 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7b65d-113b-4964-8e37-c27599a8c956 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:48.809333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e921977-8443-4ee9-9ab1-3242b34c5778 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.810018 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e921977-8443-4ee9-9ab1-3242b34c5778 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.810293 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e921977-8443-4ee9-9ab1-3242b34c5778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.849780 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-7e921977-8443-4ee9-9ab1-3242b34c5778 None None] MFA Rules not processed for user `90101c531f0d4e938d16a96da8191c24`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:48.851247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7e921977-8443-4ee9-9ab1-3242b34c5778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:48.896299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f0499-9d00-48af-aba7-97f60c53ad5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:48.897518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f0499-9d00-48af-aba7-97f60c53ad5d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:48.898034 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f0499-9d00-48af-aba7-97f60c53ad5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:48.898415 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f0499-9d00-48af-aba7-97f60c53ad5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:48.899350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f0499-9d00-48af-aba7-97f60c53ad5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:48.901668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f0499-9d00-48af-aba7-97f60c53ad5d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:48.953189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1128/2270] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:38:48 2022] POST /identity/v3/projects => generated 385 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:48.955395 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:48.956479 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f0499-9d00-48af-aba7-97f60c53ad5d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:48.968477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1143/2271] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:48.983245 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e921977-8443-4ee9-9ab1-3242b34c5778 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:48.989183 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7e921977-8443-4ee9-9ab1-3242b34c5778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:49.079636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f0499-9d00-48af-aba7-97f60c53ad5d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:49.085563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ad127-d1e0-4f75-acd4-73d5db9f9fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:49.086514 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ad127-d1e0-4f75-acd4-73d5db9f9fe4 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:49.087727 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ad127-d1e0-4f75-acd4-73d5db9f9fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:49.088899 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ad127-d1e0-4f75-acd4-73d5db9f9fe4 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:49.094028 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ad127-d1e0-4f75-acd4-73d5db9f9fe4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:49.100420 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ad127-d1e0-4f75-acd4-73d5db9f9fe4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:49.145922 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e921977-8443-4ee9-9ab1-3242b34c5778 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:49.161433 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1129/2272] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:38:48 2022] POST /identity/v3/users => generated 299 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:49.168784 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ad127-d1e0-4f75-acd4-73d5db9f9fe4 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:49.178440 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ad127-d1e0-4f75-acd4-73d5db9f9fe4 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:49.187690 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f366c1-4feb-4519-8f6e-8d1e9279e6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:49.188378 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f366c1-4feb-4519-8f6e-8d1e9279e6bb admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:49.188674 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f366c1-4feb-4519-8f6e-8d1e9279e6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:49.188961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f366c1-4feb-4519-8f6e-8d1e9279e6bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:49.190306 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f366c1-4feb-4519-8f6e-8d1e9279e6bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:49.191963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f366c1-4feb-4519-8f6e-8d1e9279e6bb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:49.206833 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1130/2273] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:49.212003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f366c1-4feb-4519-8f6e-8d1e9279e6bb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:49.223841 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f366c1-4feb-4519-8f6e-8d1e9279e6bb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:49.232553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e627844-5f33-40a9-81b8-03b73af6e16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:49.233504 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e627844-5f33-40a9-81b8-03b73af6e16c admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:49.233894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e627844-5f33-40a9-81b8-03b73af6e16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:49.234275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e627844-5f33-40a9-81b8-03b73af6e16c admin admin] PATH_INFO: `/v3/projects/cbf964f782034092a884132ff33c0c63/users/82b971a8820746df9253409dde912ee4/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:49.254615 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e627844-5f33-40a9-81b8-03b73af6e16c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf964f782034092a884132ff33c0c63, user_id=82b971a8820746df9253409dde912ee4, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:49.257817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e627844-5f33-40a9-81b8-03b73af6e16c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:49.278074 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521feb4d-db66-4d19-a21c-57a9210b3d2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:49.278856 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521feb4d-db66-4d19-a21c-57a9210b3d2b service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:49.279268 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521feb4d-db66-4d19-a21c-57a9210b3d2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:49.279637 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521feb4d-db66-4d19-a21c-57a9210b3d2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:49.286834 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1131/2274] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:49 2022] PUT /identity/v3/projects/cbf964f782034092a884132ff33c0c63/users/82b971a8820746df9253409dde912ee4/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:49.290767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e627844-5f33-40a9-81b8-03b73af6e16c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:49.369002 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:49.414850 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e627844-5f33-40a9-81b8-03b73af6e16c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:49.431441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0720baf-ba45-4b2f-a0f0-807906ece5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:49.432276 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0720baf-ba45-4b2f-a0f0-807906ece5ad admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:49.432641 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0720baf-ba45-4b2f-a0f0-807906ece5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:49.433318 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0720baf-ba45-4b2f-a0f0-807906ece5ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:49.434015 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0720baf-ba45-4b2f-a0f0-807906ece5ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:49.434704 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521feb4d-db66-4d19-a21c-57a9210b3d2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:49.437553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0720baf-ba45-4b2f-a0f0-807906ece5ad admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:49.437702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521feb4d-db66-4d19-a21c-57a9210b3d2b service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:49.455068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1132/2275] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:49 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:38:49.459618 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0720baf-ba45-4b2f-a0f0-807906ece5ad admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:49.471234 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1144/2276] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:49.476728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0720baf-ba45-4b2f-a0f0-807906ece5ad admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:49.513896 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0fa6b-dfcc-4f80-af51-cb52dfc50dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:49.514440 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0fa6b-dfcc-4f80-af51-cb52dfc50dba admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:49.514738 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0fa6b-dfcc-4f80-af51-cb52dfc50dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:49.515054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0fa6b-dfcc-4f80-af51-cb52dfc50dba admin admin] PATH_INFO: `/v3/projects/cbf964f782034092a884132ff33c0c63/users/82b971a8820746df9253409dde912ee4/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:49.517829 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0fa6b-dfcc-4f80-af51-cb52dfc50dba admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf964f782034092a884132ff33c0c63, user_id=82b971a8820746df9253409dde912ee4, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:49.520098 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0fa6b-dfcc-4f80-af51-cb52dfc50dba admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:49.540427 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1133/2277] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:49 2022] PUT /identity/v3/projects/cbf964f782034092a884132ff33c0c63/users/82b971a8820746df9253409dde912ee4/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:49.552259 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68f2ae-9573-427a-b9b8-9fc00780dde2 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:49.552560 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68f2ae-9573-427a-b9b8-9fc00780dde2 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:49.552853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68f2ae-9573-427a-b9b8-9fc00780dde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:49.609634 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-af68f2ae-9573-427a-b9b8-9fc00780dde2 None None] MFA Rules not processed for user `82b971a8820746df9253409dde912ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:49.610699 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-af68f2ae-9573-427a-b9b8-9fc00780dde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:49.702696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:49.729006 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1145/2278] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:38:49.794666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0fa6b-dfcc-4f80-af51-cb52dfc50dba admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:49.796548 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a0d0fa6b-dfcc-4f80-af51-cb52dfc50dba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:49.876972 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0fa6b-dfcc-4f80-af51-cb52dfc50dba admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:49.881061 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03042ed-912e-4e1c-bc56-b61d6444f055 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:49.881717 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03042ed-912e-4e1c-bc56-b61d6444f055 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:49.882000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03042ed-912e-4e1c-bc56-b61d6444f055 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:49.882312 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03042ed-912e-4e1c-bc56-b61d6444f055 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:49.893990 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03042ed-912e-4e1c-bc56-b61d6444f055 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:49.895076 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03042ed-912e-4e1c-bc56-b61d6444f055 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:49.915071 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1134/2279] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:50.046472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68f2ae-9573-427a-b9b8-9fc00780dde2 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:50.109649 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68f2ae-9573-427a-b9b8-9fc00780dde2 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:50.114038 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46bde4-038e-4e57-9ac2-c64efce4ea79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:50.114904 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46bde4-038e-4e57-9ac2-c64efce4ea79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:50.115222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46bde4-038e-4e57-9ac2-c64efce4ea79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:50.115501 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46bde4-038e-4e57-9ac2-c64efce4ea79 admin admin] PATH_INFO: `/v3/users/7faddb9f17084b0186777e17d610628a` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:50.116677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46bde4-038e-4e57-9ac2-c64efce4ea79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7faddb9f17084b0186777e17d610628a)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:50.118596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46bde4-038e-4e57-9ac2-c64efce4ea79 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:50.169586 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-2e46bde4-038e-4e57-9ac2-c64efce4ea79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7faddb9f17084b0186777e17d610628a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:50.182793 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-2e46bde4-038e-4e57-9ac2-c64efce4ea79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7faddb9f17084b0186777e17d610628a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:50.191425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-2e46bde4-038e-4e57-9ac2-c64efce4ea79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7faddb9f17084b0186777e17d610628a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:50.199753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-2e46bde4-038e-4e57-9ac2-c64efce4ea79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7faddb9f17084b0186777e17d610628a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:50.200724 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-2e46bde4-038e-4e57-9ac2-c64efce4ea79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7faddb9f17084b0186777e17d610628a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:50.207555 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1146/2280] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:38:50 2022] DELETE /identity/v3/users/7faddb9f17084b0186777e17d610628a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:38:50.212291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236aec1-654f-49ac-bae7-49cf570c3413 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:50.212811 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236aec1-654f-49ac-bae7-49cf570c3413 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:50.212908 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236aec1-654f-49ac-bae7-49cf570c3413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:50.231546 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9236aec1-654f-49ac-bae7-49cf570c3413 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:50.233062 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9236aec1-654f-49ac-bae7-49cf570c3413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:50.300702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1135/2281] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:50.316198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46bde4-038e-4e57-9ac2-c64efce4ea79 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:50.317801 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2e46bde4-038e-4e57-9ac2-c64efce4ea79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:50.382795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46bde4-038e-4e57-9ac2-c64efce4ea79 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:50.386886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c8a45-d11d-4ae4-b322-4dea8761c1f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:50.388237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c8a45-d11d-4ae4-b322-4dea8761c1f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:50.388532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c8a45-d11d-4ae4-b322-4dea8761c1f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:50.388818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c8a45-d11d-4ae4-b322-4dea8761c1f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:50.398834 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c8a45-d11d-4ae4-b322-4dea8761c1f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:50.400320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c8a45-d11d-4ae4-b322-4dea8761c1f0 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:50.411221 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1147/2282] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:50.802339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236aec1-654f-49ac-bae7-49cf570c3413 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:50.803917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9236aec1-654f-49ac-bae7-49cf570c3413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:50.817848 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236aec1-654f-49ac-bae7-49cf570c3413 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:50.825411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965e1fa-c530-4e46-846e-9a28c9759bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:50.826255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965e1fa-c530-4e46-846e-9a28c9759bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:50.826543 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965e1fa-c530-4e46-846e-9a28c9759bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:50.826840 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965e1fa-c530-4e46-846e-9a28c9759bdc admin admin] PATH_INFO: `/v3/projects/330603d973f347229d00af15d8357040` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:50.828098 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965e1fa-c530-4e46-846e-9a28c9759bdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=330603d973f347229d00af15d8357040)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:50.830017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965e1fa-c530-4e46-846e-9a28c9759bdc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:50.894493 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6965e1fa-c530-4e46-846e-9a28c9759bdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '330603d973f347229d00af15d8357040', 'request_id': 'req-6965e1fa-c530-4e46-846e-9a28c9759bdc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:50.898115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:50.917040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6965e1fa-c530-4e46-846e-9a28c9759bdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '330603d973f347229d00af15d8357040', 'request_id': 'req-6965e1fa-c530-4e46-846e-9a28c9759bdc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:50.926149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6965e1fa-c530-4e46-846e-9a28c9759bdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '330603d973f347229d00af15d8357040', 'request_id': 'req-6965e1fa-c530-4e46-846e-9a28c9759bdc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:50.930590 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c8a45-d11d-4ae4-b322-4dea8761c1f0 service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:50.932116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-a22c8a45-d11d-4ae4-b322-4dea8761c1f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:50.943082 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6965e1fa-c530-4e46-846e-9a28c9759bdc admin admin] The token cache is being invalidate because project 330603d973f347229d00af15d8357040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:50.943446 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6965e1fa-c530-4e46-846e-9a28c9759bdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:50.944153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6965e1fa-c530-4e46-846e-9a28c9759bdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:50.945824 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1136/2283] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:50 2022] DELETE /identity/v3/projects/330603d973f347229d00af15d8357040 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:50.946174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c8a45-d11d-4ae4-b322-4dea8761c1f0 service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:50.950215 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965e1fa-c530-4e46-846e-9a28c9759bdc admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:50.951625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6965e1fa-c530-4e46-846e-9a28c9759bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:50.956415 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:51.023654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23a7ca-189e-4993-8925-c02dd42bbbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:51.024655 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23a7ca-189e-4993-8925-c02dd42bbbe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:51.024943 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23a7ca-189e-4993-8925-c02dd42bbbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:51.025228 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23a7ca-189e-4993-8925-c02dd42bbbe8 admin admin] PATH_INFO: `/v3/users/34f24b9b6f2d45c4907ce8cb419ab4a4` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:51.026663 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a23a7ca-189e-4993-8925-c02dd42bbbe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34f24b9b6f2d45c4907ce8cb419ab4a4)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:51.029415 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a23a7ca-189e-4993-8925-c02dd42bbbe8 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:51.037448 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965e1fa-c530-4e46-846e-9a28c9759bdc admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:51.042716 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d993de-2dc6-429f-9b9b-22b9de0ed837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:51.044033 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d993de-2dc6-429f-9b9b-22b9de0ed837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:51.044339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d993de-2dc6-429f-9b9b-22b9de0ed837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:51.044742 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d993de-2dc6-429f-9b9b-22b9de0ed837 admin admin] PATH_INFO: `/v3/users/82b971a8820746df9253409dde912ee4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:51.046949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d993de-2dc6-429f-9b9b-22b9de0ed837 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82b971a8820746df9253409dde912ee4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:51.049421 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d993de-2dc6-429f-9b9b-22b9de0ed837 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:51.086293 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7a23a7ca-189e-4993-8925-c02dd42bbbe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34f24b9b6f2d45c4907ce8cb419ab4a4'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.101048 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7a23a7ca-189e-4993-8925-c02dd42bbbe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34f24b9b6f2d45c4907ce8cb419ab4a4'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.110227 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7a23a7ca-189e-4993-8925-c02dd42bbbe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34f24b9b6f2d45c4907ce8cb419ab4a4'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.119401 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7a23a7ca-189e-4993-8925-c02dd42bbbe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34f24b9b6f2d45c4907ce8cb419ab4a4'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.120108 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7a23a7ca-189e-4993-8925-c02dd42bbbe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34f24b9b6f2d45c4907ce8cb419ab4a4'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.124958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-17d993de-2dc6-429f-9b9b-22b9de0ed837 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82b971a8820746df9253409dde912ee4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.131075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1148/2284] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:38:50 2022] DELETE /identity/v3/users/34f24b9b6f2d45c4907ce8cb419ab4a4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:51.133638 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ff50d-ceee-4483-bb72-e87d0873d1ca None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:51.133638 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ff50d-ceee-4483-bb72-e87d0873d1ca None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:51.134015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ff50d-ceee-4483-bb72-e87d0873d1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:51.134132 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-17d993de-2dc6-429f-9b9b-22b9de0ed837 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82b971a8820746df9253409dde912ee4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.146258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-17d993de-2dc6-429f-9b9b-22b9de0ed837 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82b971a8820746df9253409dde912ee4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.147059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-17d993de-2dc6-429f-9b9b-22b9de0ed837 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82b971a8820746df9253409dde912ee4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.147913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-17d993de-2dc6-429f-9b9b-22b9de0ed837 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82b971a8820746df9253409dde912ee4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.154570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-c77ff50d-ceee-4483-bb72-e87d0873d1ca None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:51.155972 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-c77ff50d-ceee-4483-bb72-e87d0873d1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:51.172158 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1137/2285] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:38:50 2022] DELETE /identity/v3/users/82b971a8820746df9253409dde912ee4 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:38:51.237103 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1149/2286] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:51.248476 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d993de-2dc6-429f-9b9b-22b9de0ed837 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:51.250283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-17d993de-2dc6-429f-9b9b-22b9de0ed837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:51.313497 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d993de-2dc6-429f-9b9b-22b9de0ed837 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:51.320087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ed342-da67-44e8-b6ec-6469d3addd5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:51.320767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ed342-da67-44e8-b6ec-6469d3addd5d service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:51.321097 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ed342-da67-44e8-b6ec-6469d3addd5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:51.321328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ed342-da67-44e8-b6ec-6469d3addd5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:51.347912 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:51.423761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ed342-da67-44e8-b6ec-6469d3addd5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:51.425928 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ed342-da67-44e8-b6ec-6469d3addd5d service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:51.436710 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1138/2287] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:38:51.513430 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ff50d-ceee-4483-bb72-e87d0873d1ca None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:51.514900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-c77ff50d-ceee-4483-bb72-e87d0873d1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:51.530995 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ff50d-ceee-4483-bb72-e87d0873d1ca None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:51.538069 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a5d97-69a7-41c8-bb2f-4aff8ea11629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:51.539111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a5d97-69a7-41c8-bb2f-4aff8ea11629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:51.539863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a5d97-69a7-41c8-bb2f-4aff8ea11629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:51.539863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a5d97-69a7-41c8-bb2f-4aff8ea11629 admin admin] PATH_INFO: `/v3/projects/cbf964f782034092a884132ff33c0c63` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:51.541740 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a5d97-69a7-41c8-bb2f-4aff8ea11629 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbf964f782034092a884132ff33c0c63)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:51.544887 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a5d97-69a7-41c8-bb2f-4aff8ea11629 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:51.594882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:51.634296 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-869a5d97-69a7-41c8-bb2f-4aff8ea11629 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbf964f782034092a884132ff33c0c63', 'request_id': 'req-869a5d97-69a7-41c8-bb2f-4aff8ea11629'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.644594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-869a5d97-69a7-41c8-bb2f-4aff8ea11629 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbf964f782034092a884132ff33c0c63', 'request_id': 'req-869a5d97-69a7-41c8-bb2f-4aff8ea11629'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.663218 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-869a5d97-69a7-41c8-bb2f-4aff8ea11629 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbf964f782034092a884132ff33c0c63', 'request_id': 'req-869a5d97-69a7-41c8-bb2f-4aff8ea11629'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.679501 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-869a5d97-69a7-41c8-bb2f-4aff8ea11629 admin admin] The token cache is being invalidate because project cbf964f782034092a884132ff33c0c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:51.679916 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-869a5d97-69a7-41c8-bb2f-4aff8ea11629 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.680753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-869a5d97-69a7-41c8-bb2f-4aff8ea11629 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.686261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1150/2288] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:51 2022] DELETE /identity/v3/projects/cbf964f782034092a884132ff33c0c63 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:51.689173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c5b2b-cffc-4f37-8e00-4cf4b09b7f66 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:51.689474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c5b2b-cffc-4f37-8e00-4cf4b09b7f66 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:51.689795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c5b2b-cffc-4f37-8e00-4cf4b09b7f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:51.701009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a5d97-69a7-41c8-bb2f-4aff8ea11629 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:51.719378 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-869a5d97-69a7-41c8-bb2f-4aff8ea11629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:51.756517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3b2c5b2b-cffc-4f37-8e00-4cf4b09b7f66 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:51.757746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3b2c5b2b-cffc-4f37-8e00-4cf4b09b7f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:51.854343 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a5d97-69a7-41c8-bb2f-4aff8ea11629 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:51.860166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f73e1-7fee-4407-a41b-854602370524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:51.861375 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f73e1-7fee-4407-a41b-854602370524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:51.861929 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f73e1-7fee-4407-a41b-854602370524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:51.862380 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f73e1-7fee-4407-a41b-854602370524 admin admin] PATH_INFO: `/v3/projects/f7bc6b504b764fe6b3f892e2a8a88a05` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:51.862517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1139/2289] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:51.863988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f73e1-7fee-4407-a41b-854602370524 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7bc6b504b764fe6b3f892e2a8a88a05)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:51.866821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f73e1-7fee-4407-a41b-854602370524 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:51.871548 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dc746-b09b-4d95-9409-e3b9195b69b3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:51.872803 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dc746-b09b-4d95-9409-e3b9195b69b3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:51.873153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dc746-b09b-4d95-9409-e3b9195b69b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:51.899194 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1a1dc746-b09b-4d95-9409-e3b9195b69b3 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:51.900576 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1a1dc746-b09b-4d95-9409-e3b9195b69b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:51.911690 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1140/2290] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:38:51.916520 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1dc746-b09b-4d95-9409-e3b9195b69b3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:51.935127 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e80f73e1-7fee-4407-a41b-854602370524 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7bc6b504b764fe6b3f892e2a8a88a05', 'request_id': 'req-e80f73e1-7fee-4407-a41b-854602370524'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.943843 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1dc746-b09b-4d95-9409-e3b9195b69b3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:51.950201 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e80f73e1-7fee-4407-a41b-854602370524 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7bc6b504b764fe6b3f892e2a8a88a05', 'request_id': 'req-e80f73e1-7fee-4407-a41b-854602370524'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.963128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e80f73e1-7fee-4407-a41b-854602370524 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7bc6b504b764fe6b3f892e2a8a88a05', 'request_id': 'req-e80f73e1-7fee-4407-a41b-854602370524'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.977860 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e80f73e1-7fee-4407-a41b-854602370524 admin admin] The token cache is being invalidate because project f7bc6b504b764fe6b3f892e2a8a88a05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:51.978210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e80f73e1-7fee-4407-a41b-854602370524 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.978864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-e80f73e1-7fee-4407-a41b-854602370524 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:51.982199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1151/2291] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:51 2022] DELETE /identity/v3/projects/f7bc6b504b764fe6b3f892e2a8a88a05 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:51.987586 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3783359-7552-4d92-a08f-74c49fdd2bf5 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:51.987902 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3783359-7552-4d92-a08f-74c49fdd2bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:51.988168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3783359-7552-4d92-a08f-74c49fdd2bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.010035 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-f3783359-7552-4d92-a08f-74c49fdd2bf5 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:52.011369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-f3783359-7552-4d92-a08f-74c49fdd2bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:52.019635 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81c637-6603-4ef5-966a-f3bc5b77c097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:52.020311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81c637-6603-4ef5-966a-f3bc5b77c097 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.020603 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81c637-6603-4ef5-966a-f3bc5b77c097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.020887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81c637-6603-4ef5-966a-f3bc5b77c097 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.021616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81c637-6603-4ef5-966a-f3bc5b77c097 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:38:52.021930 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81c637-6603-4ef5-966a-f3bc5b77c097 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:52.023558 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81c637-6603-4ef5-966a-f3bc5b77c097 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:52.036380 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1141/2292] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:38:52.040623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81c637-6603-4ef5-966a-f3bc5b77c097 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:52.042381 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3a81c637-6603-4ef5-966a-f3bc5b77c097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:52.057174 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81c637-6603-4ef5-966a-f3bc5b77c097 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:52.060955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:52.065268 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380ee15-e406-48f1-b369-0132c8459463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:52.065942 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380ee15-e406-48f1-b369-0132c8459463 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.066264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380ee15-e406-48f1-b369-0132c8459463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.066584 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380ee15-e406-48f1-b369-0132c8459463 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.067326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380ee15-e406-48f1-b369-0132c8459463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:52.069627 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380ee15-e406-48f1-b369-0132c8459463 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:52.084250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1152/2293] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:52.091930 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8189e-3591-452c-907e-af52851fc3e4 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.092224 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8189e-3591-452c-907e-af52851fc3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.092510 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8189e-3591-452c-907e-af52851fc3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.104212 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1142/2294] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:38:52 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:38:52.108842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380ee15-e406-48f1-b369-0132c8459463 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:52.111568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-2df8189e-3591-452c-907e-af52851fc3e4 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:52.112779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2df8189e-3591-452c-907e-af52851fc3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:52.130617 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:52.180791 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380ee15-e406-48f1-b369-0132c8459463 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:52.184983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1153/2295] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:38:52.186368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c40f9b-9639-46a8-aa96-7d4c725a7aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:52.187083 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c40f9b-9639-46a8-aa96-7d4c725a7aed admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.187373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c40f9b-9639-46a8-aa96-7d4c725a7aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.187653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c40f9b-9639-46a8-aa96-7d4c725a7aed admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.188347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c40f9b-9639-46a8-aa96-7d4c725a7aed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:52.189458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8189e-3591-452c-907e-af52851fc3e4 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:52.190397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c40f9b-9639-46a8-aa96-7d4c725a7aed admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:52.200584 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8189e-3591-452c-907e-af52851fc3e4 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:52.206039 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5c21a-53ac-4cc1-9c78-bc26724b242a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:52.206703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5c21a-53ac-4cc1-9c78-bc26724b242a admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.207028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5c21a-53ac-4cc1-9c78-bc26724b242a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.207319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5c21a-53ac-4cc1-9c78-bc26724b242a admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.208062 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba5c21a-53ac-4cc1-9c78-bc26724b242a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:38:52.208378 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba5c21a-53ac-4cc1-9c78-bc26724b242a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:52.209993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba5c21a-53ac-4cc1-9c78-bc26724b242a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:52.212207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1143/2296] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:38:52 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:52.216897 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c40f9b-9639-46a8-aa96-7d4c725a7aed admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:52.227243 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c40f9b-9639-46a8-aa96-7d4c725a7aed admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:52.227389 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1154/2297] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:38:52.231776 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5c21a-53ac-4cc1-9c78-bc26724b242a admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:52.232524 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d101e6-ce82-465d-b255-f0029764e87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:52.233203 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d101e6-ce82-465d-b255-f0029764e87a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.233491 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d101e6-ce82-465d-b255-f0029764e87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.233774 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d101e6-ce82-465d-b255-f0029764e87a admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.234503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d101e6-ce82-465d-b255-f0029764e87a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:52.236124 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d101e6-ce82-465d-b255-f0029764e87a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:52.242538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5c21a-53ac-4cc1-9c78-bc26724b242a admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:52.248401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1144/2298] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:38:52.248938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40073dbb-9d81-422e-9974-3572fa529e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:52.249613 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40073dbb-9d81-422e-9974-3572fa529e1a admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.249935 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40073dbb-9d81-422e-9974-3572fa529e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.250365 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40073dbb-9d81-422e-9974-3572fa529e1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.251182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40073dbb-9d81-422e-9974-3572fa529e1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:52.252367 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d101e6-ce82-465d-b255-f0029764e87a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:52.253494 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40073dbb-9d81-422e-9974-3572fa529e1a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:52.269133 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d101e6-ce82-465d-b255-f0029764e87a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:52.275010 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf9603-95e3-4bde-99bc-94757bef32ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:52.275708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf9603-95e3-4bde-99bc-94757bef32ac admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.275967 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf9603-95e3-4bde-99bc-94757bef32ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.276244 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf9603-95e3-4bde-99bc-94757bef32ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.277201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf9603-95e3-4bde-99bc-94757bef32ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:52.278611 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf9603-95e3-4bde-99bc-94757bef32ac admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:52.284399 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1155/2299] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:38:52 2022] POST /identity/v3/projects => generated 391 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:38:52.286139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40073dbb-9d81-422e-9974-3572fa529e1a admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:52.288771 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-40073dbb-9d81-422e-9974-3572fa529e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:52.291662 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1145/2300] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:38:52.292570 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf9603-95e3-4bde-99bc-94757bef32ac admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:52.303434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf9603-95e3-4bde-99bc-94757bef32ac admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:52.361696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40073dbb-9d81-422e-9974-3572fa529e1a admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:52.369554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0050f-00f3-4ccb-989f-371be781c79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:52.370455 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0050f-00f3-4ccb-989f-371be781c79f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.371348 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0050f-00f3-4ccb-989f-371be781c79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.371637 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0050f-00f3-4ccb-989f-371be781c79f admin admin] PATH_INFO: `/v3/users/90101c531f0d4e938d16a96da8191c24` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.372915 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0050f-00f3-4ccb-989f-371be781c79f admin admin] RBAC: Authorizing `identity:delete_user(user_id=90101c531f0d4e938d16a96da8191c24)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:52.374868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0050f-00f3-4ccb-989f-371be781c79f admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:52.376708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf54059-1c22-4238-ba4b-dd88cd31fa9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:52.378010 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf54059-1c22-4238-ba4b-dd88cd31fa9d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.378409 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf54059-1c22-4238-ba4b-dd88cd31fa9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.378907 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf54059-1c22-4238-ba4b-dd88cd31fa9d admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.379842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf54059-1c22-4238-ba4b-dd88cd31fa9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:52.382267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf54059-1c22-4238-ba4b-dd88cd31fa9d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:52.408330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1146/2301] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:38:52 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:52.408330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf54059-1c22-4238-ba4b-dd88cd31fa9d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:52.421258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf54059-1c22-4238-ba4b-dd88cd31fa9d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:52.426261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22035705-7fab-46b7-9753-e9f94733691e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:52.427211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22035705-7fab-46b7-9753-e9f94733691e admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.427479 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22035705-7fab-46b7-9753-e9f94733691e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.427772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22035705-7fab-46b7-9753-e9f94733691e admin admin] PATH_INFO: `/v3/projects/adc5f1c72d7a4bc3ad31ec8b0a33b1d4/users/d4a94cdbdb324c43ba2cf0243df035d3/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.438163 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-02a0050f-00f3-4ccb-989f-371be781c79f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90101c531f0d4e938d16a96da8191c24'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:52.443088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:52.444109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22035705-7fab-46b7-9753-e9f94733691e admin admin] RBAC: Authorizing `identity:create_grant(project_id=adc5f1c72d7a4bc3ad31ec8b0a33b1d4, user_id=d4a94cdbdb324c43ba2cf0243df035d3, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:52.446908 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22035705-7fab-46b7-9753-e9f94733691e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:52.455499 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-02a0050f-00f3-4ccb-989f-371be781c79f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90101c531f0d4e938d16a96da8191c24'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:52.462259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1147/2302] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:52 2022] PUT /identity/v3/projects/adc5f1c72d7a4bc3ad31ec8b0a33b1d4/users/d4a94cdbdb324c43ba2cf0243df035d3/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:52.463883 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-02a0050f-00f3-4ccb-989f-371be781c79f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90101c531f0d4e938d16a96da8191c24'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:52.467828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22035705-7fab-46b7-9753-e9f94733691e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:52.472692 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-02a0050f-00f3-4ccb-989f-371be781c79f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90101c531f0d4e938d16a96da8191c24'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:52.473371 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-02a0050f-00f3-4ccb-989f-371be781c79f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90101c531f0d4e938d16a96da8191c24'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:52.479414 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1156/2303] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:38:52 2022] DELETE /identity/v3/users/90101c531f0d4e938d16a96da8191c24 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:52.481614 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c4441-8162-457d-a573-e176a309263e None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.481911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c4441-8162-457d-a573-e176a309263e None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.482196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c4441-8162-457d-a573-e176a309263e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.510183 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-343c4441-8162-457d-a573-e176a309263e None None] MFA Rules not processed for user `d4a94cdbdb324c43ba2cf0243df035d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:52.511745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-343c4441-8162-457d-a573-e176a309263e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:52.521980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:52.536586 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22035705-7fab-46b7-9753-e9f94733691e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:52.585526 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1157/2304] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:52.586849 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6833d-8d4d-4920-86ef-72dafa5b8ca4 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.587174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6833d-8d4d-4920-86ef-72dafa5b8ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.587465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6833d-8d4d-4920-86ef-72dafa5b8ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.603776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ca3bb-82f3-45c8-a033-7d382c6cbd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:52.604490 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ca3bb-82f3-45c8-a033-7d382c6cbd8c admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.604809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ca3bb-82f3-45c8-a033-7d382c6cbd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.605148 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ca3bb-82f3-45c8-a033-7d382c6cbd8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.605951 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ca3bb-82f3-45c8-a033-7d382c6cbd8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:52.606139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-dea6833d-8d4d-4920-86ef-72dafa5b8ca4 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:52.607387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-dea6833d-8d4d-4920-86ef-72dafa5b8ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:52.607607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ca3bb-82f3-45c8-a033-7d382c6cbd8c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:52.623567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1158/2305] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:52.623753 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1148/2306] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:52.623865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6833d-8d4d-4920-86ef-72dafa5b8ca4 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:52.623865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-dea6833d-8d4d-4920-86ef-72dafa5b8ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:52.624051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ca3bb-82f3-45c8-a033-7d382c6cbd8c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:52.624051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-248ca3bb-82f3-45c8-a033-7d382c6cbd8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:52.636070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ca3bb-82f3-45c8-a033-7d382c6cbd8c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:52.641630 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a6b86-466c-44d6-8547-bc07caa15750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:52.642299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a6b86-466c-44d6-8547-bc07caa15750 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.642591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a6b86-466c-44d6-8547-bc07caa15750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.642890 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a6b86-466c-44d6-8547-bc07caa15750 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.643696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a6b86-466c-44d6-8547-bc07caa15750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:52.645293 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a6b86-466c-44d6-8547-bc07caa15750 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:52.656396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1149/2307] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:52.657581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a6b86-466c-44d6-8547-bc07caa15750 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:52.659629 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b58a6b86-466c-44d6-8547-bc07caa15750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:52.684864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6833d-8d4d-4920-86ef-72dafa5b8ca4 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:52.688588 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85112e48-4821-4cb5-b414-4f16e0482efa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:52.689232 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85112e48-4821-4cb5-b414-4f16e0482efa service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.689519 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85112e48-4821-4cb5-b414-4f16e0482efa service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.689803 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85112e48-4821-4cb5-b414-4f16e0482efa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.698315 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85112e48-4821-4cb5-b414-4f16e0482efa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:52.700435 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85112e48-4821-4cb5-b414-4f16e0482efa service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:52.709623 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1159/2308] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:52.710223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85112e48-4821-4cb5-b414-4f16e0482efa service cinder] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:52.723054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a6b86-466c-44d6-8547-bc07caa15750 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:52.727115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c111acf-41ae-4e92-a9c6-33b960864c60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:52.727335 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85112e48-4821-4cb5-b414-4f16e0482efa service cinder] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:52.727779 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c111acf-41ae-4e92-a9c6-33b960864c60 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.728064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c111acf-41ae-4e92-a9c6-33b960864c60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.728344 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c111acf-41ae-4e92-a9c6-33b960864c60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.732386 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1c062-d28d-448f-bee4-a4d671e2fe11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:52.733285 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1c062-d28d-448f-bee4-a4d671e2fe11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.739255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1c062-d28d-448f-bee4-a4d671e2fe11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.739255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1c062-d28d-448f-bee4-a4d671e2fe11 admin admin] PATH_INFO: `/v3/projects/d445ce0afdb6479f904809eb5787cd2f/users/6de25a66435b4095b76d8f603a7a2759/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.739611 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c111acf-41ae-4e92-a9c6-33b960864c60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:52.739848 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c111acf-41ae-4e92-a9c6-33b960864c60 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:52.750124 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1150/2309] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:52.751095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c111acf-41ae-4e92-a9c6-33b960864c60 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:52.752437 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1c062-d28d-448f-bee4-a4d671e2fe11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d445ce0afdb6479f904809eb5787cd2f, user_id=6de25a66435b4095b76d8f603a7a2759, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:52.752858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1c111acf-41ae-4e92-a9c6-33b960864c60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:52.755333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1c062-d28d-448f-bee4-a4d671e2fe11 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:52.768296 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c111acf-41ae-4e92-a9c6-33b960864c60 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:52.772438 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1160/2310] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:52 2022] PUT /identity/v3/projects/d445ce0afdb6479f904809eb5787cd2f/users/6de25a66435b4095b76d8f603a7a2759/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:52.786249 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06492a-a65b-48e4-8b48-a15449255da0 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.786613 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06492a-a65b-48e4-8b48-a15449255da0 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.786818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06492a-a65b-48e4-8b48-a15449255da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.818663 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-8f06492a-a65b-48e4-8b48-a15449255da0 None None] MFA Rules not processed for user `6de25a66435b4095b76d8f603a7a2759`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:52.818663 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8f06492a-a65b-48e4-8b48-a15449255da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:52.844994 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:52.870358 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb493f73-6efa-4969-800d-5d3aa3496e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:52.872145 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb493f73-6efa-4969-800d-5d3aa3496e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:52.872680 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb493f73-6efa-4969-800d-5d3aa3496e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:52.873273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb493f73-6efa-4969-800d-5d3aa3496e20 admin admin] PATH_INFO: `/v3/users/d4a94cdbdb324c43ba2cf0243df035d3` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:52.875599 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb493f73-6efa-4969-800d-5d3aa3496e20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4a94cdbdb324c43ba2cf0243df035d3)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:52.879177 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb493f73-6efa-4969-800d-5d3aa3496e20 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:52.915466 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1161/2311] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:52.917109 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:52.926641 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f06492a-a65b-48e4-8b48-a15449255da0 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:52.928195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8f06492a-a65b-48e4-8b48-a15449255da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:52.948791 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-eb493f73-6efa-4969-800d-5d3aa3496e20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4a94cdbdb324c43ba2cf0243df035d3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:52.971950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-eb493f73-6efa-4969-800d-5d3aa3496e20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4a94cdbdb324c43ba2cf0243df035d3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:52.986392 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-eb493f73-6efa-4969-800d-5d3aa3496e20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4a94cdbdb324c43ba2cf0243df035d3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:52.987135 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-eb493f73-6efa-4969-800d-5d3aa3496e20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4a94cdbdb324c43ba2cf0243df035d3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:52.987877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-eb493f73-6efa-4969-800d-5d3aa3496e20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4a94cdbdb324c43ba2cf0243df035d3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:53.020997 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1151/2312] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:38:52 2022] DELETE /identity/v3/users/d4a94cdbdb324c43ba2cf0243df035d3 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:53.025258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326690f-19f4-4ab0-80ed-f3250d0d16f2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:53.025683 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326690f-19f4-4ab0-80ed-f3250d0d16f2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:53.026358 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326690f-19f4-4ab0-80ed-f3250d0d16f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:53.056902 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f06492a-a65b-48e4-8b48-a15449255da0 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:53.058386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b326690f-19f4-4ab0-80ed-f3250d0d16f2 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:53.060370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b326690f-19f4-4ab0-80ed-f3250d0d16f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:53.170702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1152/2313] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:53.174881 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326690f-19f4-4ab0-80ed-f3250d0d16f2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:53.176613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b326690f-19f4-4ab0-80ed-f3250d0d16f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:53.196565 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582a4fa-eab5-4062-a8db-6d45370a146e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:53.199202 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582a4fa-eab5-4062-a8db-6d45370a146e service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:53.199919 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582a4fa-eab5-4062-a8db-6d45370a146e service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:53.200448 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582a4fa-eab5-4062-a8db-6d45370a146e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:53.200630 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326690f-19f4-4ab0-80ed-f3250d0d16f2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:53.203631 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7582a4fa-eab5-4062-a8db-6d45370a146e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:53.206326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc3bf7-7e38-431f-8956-ce4aec559613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:53.207289 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc3bf7-7e38-431f-8956-ce4aec559613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:53.207583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc3bf7-7e38-431f-8956-ce4aec559613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:53.207897 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc3bf7-7e38-431f-8956-ce4aec559613 admin admin] PATH_INFO: `/v3/projects/967d387cb4e24455bfdcea3f32d27755` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:53.209081 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc3bf7-7e38-431f-8956-ce4aec559613 admin admin] RBAC: Authorizing `identity:delete_project(project_id=967d387cb4e24455bfdcea3f32d27755)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:53.211123 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc3bf7-7e38-431f-8956-ce4aec559613 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:53.278333 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9adc3bf7-7e38-431f-8956-ce4aec559613 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '967d387cb4e24455bfdcea3f32d27755', 'request_id': 'req-9adc3bf7-7e38-431f-8956-ce4aec559613'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:53.287944 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9adc3bf7-7e38-431f-8956-ce4aec559613 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '967d387cb4e24455bfdcea3f32d27755', 'request_id': 'req-9adc3bf7-7e38-431f-8956-ce4aec559613'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:53.298153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9adc3bf7-7e38-431f-8956-ce4aec559613 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '967d387cb4e24455bfdcea3f32d27755', 'request_id': 'req-9adc3bf7-7e38-431f-8956-ce4aec559613'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:53.311052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9adc3bf7-7e38-431f-8956-ce4aec559613 admin admin] The token cache is being invalidate because project 967d387cb4e24455bfdcea3f32d27755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:53.311358 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9adc3bf7-7e38-431f-8956-ce4aec559613 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:53.311849 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582a4fa-eab5-4062-a8db-6d45370a146e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:53.312025 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-9adc3bf7-7e38-431f-8956-ce4aec559613 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:53.313480 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582a4fa-eab5-4062-a8db-6d45370a146e service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:53.314488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1153/2314] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:53 2022] DELETE /identity/v3/projects/967d387cb4e24455bfdcea3f32d27755 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:53.319056 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc3bf7-7e38-431f-8956-ce4aec559613 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:53.320554 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9adc3bf7-7e38-431f-8956-ce4aec559613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:53.324624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1162/2315] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:53.325923 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2c16e-c422-43f4-b495-b94055ef2824 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:53.326264 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2c16e-c422-43f4-b495-b94055ef2824 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:53.326557 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2c16e-c422-43f4-b495-b94055ef2824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:53.347651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-57a2c16e-c422-43f4-b495-b94055ef2824 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:53.348926 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-57a2c16e-c422-43f4-b495-b94055ef2824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:53.376411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:53.393814 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:53.423981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc3bf7-7e38-431f-8956-ce4aec559613 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:53.431509 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e9971-8ee3-4585-809d-11c5193e858b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:53.432227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e9971-8ee3-4585-809d-11c5193e858b service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:53.432606 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e9971-8ee3-4585-809d-11c5193e858b service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:53.433355 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e9971-8ee3-4585-809d-11c5193e858b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:53.439260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4c5e9971-8ee3-4585-809d-11c5193e858b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:53.465102 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1163/2316] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:53.475648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51022231-46ee-45ab-b430-a3fa4503bd24 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:53.476225 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51022231-46ee-45ab-b430-a3fa4503bd24 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:53.476670 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51022231-46ee-45ab-b430-a3fa4503bd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:53.514450 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-51022231-46ee-45ab-b430-a3fa4503bd24 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:53.516437 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-51022231-46ee-45ab-b430-a3fa4503bd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:53.530641 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1164/2317] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:53.535250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51022231-46ee-45ab-b430-a3fa4503bd24 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:53.548567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51022231-46ee-45ab-b430-a3fa4503bd24 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:53.554790 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1eada-30b7-4d78-9030-2842c4c4d5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:53.555782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1eada-30b7-4d78-9030-2842c4c4d5d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:53.556174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1eada-30b7-4d78-9030-2842c4c4d5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:53.556572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1eada-30b7-4d78-9030-2842c4c4d5d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:53.557578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1eada-30b7-4d78-9030-2842c4c4d5d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:38:53.557969 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1eada-30b7-4d78-9030-2842c4c4d5d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:53.560765 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1eada-30b7-4d78-9030-2842c4c4d5d6 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:53.564988 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e9971-8ee3-4585-809d-11c5193e858b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:53.566626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e9971-8ee3-4585-809d-11c5193e858b service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:53.577991 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1165/2318] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:38:53.581742 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1154/2319] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:53.582288 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1eada-30b7-4d78-9030-2842c4c4d5d6 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:53.595264 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1eada-30b7-4d78-9030-2842c4c4d5d6 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:53.600107 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad3bc7-02e2-4507-bdd2-0b855082edee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:53.600828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad3bc7-02e2-4507-bdd2-0b855082edee admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:53.601121 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad3bc7-02e2-4507-bdd2-0b855082edee admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:53.601441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad3bc7-02e2-4507-bdd2-0b855082edee admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:53.602219 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad3bc7-02e2-4507-bdd2-0b855082edee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:53.604422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad3bc7-02e2-4507-bdd2-0b855082edee admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:53.640497 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1166/2320] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:38:53 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:38:53.644798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e9971-8ee3-4585-809d-11c5193e858b service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:53.659087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e9971-8ee3-4585-809d-11c5193e858b service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:53.752593 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d170d-9d2d-4d92-bb38-ea2a9dce4eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:53.753277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d170d-9d2d-4d92-bb38-ea2a9dce4eb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:53.753565 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d170d-9d2d-4d92-bb38-ea2a9dce4eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:53.753846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4d170d-9d2d-4d92-bb38-ea2a9dce4eb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:53.754539 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d170d-9d2d-4d92-bb38-ea2a9dce4eb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:53.756677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4d170d-9d2d-4d92-bb38-ea2a9dce4eb3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:53.785725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1155/2321] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:38:53 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:38:53.790295 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad3bc7-02e2-4507-bdd2-0b855082edee admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:53.803080 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad3bc7-02e2-4507-bdd2-0b855082edee admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:53.807405 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5c6e8-4fa4-4fe1-88ed-01b712484f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:53.808139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5c6e8-4fa4-4fe1-88ed-01b712484f39 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:53.809119 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5c6e8-4fa4-4fe1-88ed-01b712484f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:53.809465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5c6e8-4fa4-4fe1-88ed-01b712484f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:53.810272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5c6e8-4fa4-4fe1-88ed-01b712484f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:53.811928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5c6e8-4fa4-4fe1-88ed-01b712484f39 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:53.828639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1167/2322] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:38:53.830202 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d170d-9d2d-4d92-bb38-ea2a9dce4eb3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:53.841616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4d170d-9d2d-4d92-bb38-ea2a9dce4eb3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:53.848980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589e911-4c9d-4367-aaee-a9cc63433e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:53.849903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589e911-4c9d-4367-aaee-a9cc63433e75 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:53.850193 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589e911-4c9d-4367-aaee-a9cc63433e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:53.850475 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589e911-4c9d-4367-aaee-a9cc63433e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:53.851512 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589e911-4c9d-4367-aaee-a9cc63433e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:53.853443 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589e911-4c9d-4367-aaee-a9cc63433e75 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:53.869618 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1156/2323] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:38:53.871706 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5c6e8-4fa4-4fe1-88ed-01b712484f39 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:53.884891 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5c6e8-4fa4-4fe1-88ed-01b712484f39 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:53.890304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a5483-65a3-4728-808d-bf5e428e6b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:53.891419 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a5483-65a3-4728-808d-bf5e428e6b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:53.891682 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a5483-65a3-4728-808d-bf5e428e6b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:53.892001 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a5483-65a3-4728-808d-bf5e428e6b8a admin admin] PATH_INFO: `/v3/projects/4dca85bb08ba42c5a91c02d1e5171585/users/b7b91cca223547f99f994f5267d6ac10/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:53.915957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a5483-65a3-4728-808d-bf5e428e6b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dca85bb08ba42c5a91c02d1e5171585, user_id=b7b91cca223547f99f994f5267d6ac10, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:53.919132 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a5483-65a3-4728-808d-bf5e428e6b8a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:53.936613 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1168/2324] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:38:53 2022] PUT /identity/v3/projects/4dca85bb08ba42c5a91c02d1e5171585/users/b7b91cca223547f99f994f5267d6ac10/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:38:53.946763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e8678-cc2c-42ae-9fe2-1f115a4aa790 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:53.947091 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e8678-cc2c-42ae-9fe2-1f115a4aa790 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:53.947377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e8678-cc2c-42ae-9fe2-1f115a4aa790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:53.976542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fe4e8678-cc2c-42ae-9fe2-1f115a4aa790 None None] MFA Rules not processed for user `b7b91cca223547f99f994f5267d6ac10`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:38:53.977780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fe4e8678-cc2c-42ae-9fe2-1f115a4aa790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:54.032554 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:38:54.055408 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1157/2325] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:38:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:38:54.067672 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a5483-65a3-4728-808d-bf5e428e6b8a admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:54.069685 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-dd1a5483-65a3-4728-808d-bf5e428e6b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:54.154235 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a5483-65a3-4728-808d-bf5e428e6b8a admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:54.158230 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d76bbb-3982-4f38-af92-eeec6fb454b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:54.158895 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d76bbb-3982-4f38-af92-eeec6fb454b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:54.159217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d76bbb-3982-4f38-af92-eeec6fb454b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:54.159502 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d76bbb-3982-4f38-af92-eeec6fb454b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:54.169693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d76bbb-3982-4f38-af92-eeec6fb454b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:54.171221 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d76bbb-3982-4f38-af92-eeec6fb454b9 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:54.183450 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1169/2326] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:38:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:38:54.284608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e8678-cc2c-42ae-9fe2-1f115a4aa790 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:38:54.286588 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fe4e8678-cc2c-42ae-9fe2-1f115a4aa790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:38:54.400954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e8678-cc2c-42ae-9fe2-1f115a4aa790 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:38:54.407560 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340c8c1-f0f0-4c7f-93c1-66fdad0470c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:38:54.408713 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340c8c1-f0f0-4c7f-93c1-66fdad0470c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:38:54.409207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340c8c1-f0f0-4c7f-93c1-66fdad0470c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:38:54.409596 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340c8c1-f0f0-4c7f-93c1-66fdad0470c3 admin admin] PATH_INFO: `/v3/projects/adc5f1c72d7a4bc3ad31ec8b0a33b1d4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:38:54.411583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340c8c1-f0f0-4c7f-93c1-66fdad0470c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adc5f1c72d7a4bc3ad31ec8b0a33b1d4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:38:54.413839 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340c8c1-f0f0-4c7f-93c1-66fdad0470c3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:38:54.492399 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8340c8c1-f0f0-4c7f-93c1-66fdad0470c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adc5f1c72d7a4bc3ad31ec8b0a33b1d4', 'request_id': 'req-8340c8c1-f0f0-4c7f-93c1-66fdad0470c3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:54.523347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8340c8c1-f0f0-4c7f-93c1-66fdad0470c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adc5f1c72d7a4bc3ad31ec8b0a33b1d4', 'request_id': 'req-8340c8c1-f0f0-4c7f-93c1-66fdad0470c3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:54.545825 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8340c8c1-f0f0-4c7f-93c1-66fdad0470c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adc5f1c72d7a4bc3ad31ec8b0a33b1d4', 'request_id': 'req-8340c8c1-f0f0-4c7f-93c1-66fdad0470c3'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:54.572173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8340c8c1-f0f0-4c7f-93c1-66fdad0470c3 admin admin] The token cache is being invalidate because project adc5f1c72d7a4bc3ad31ec8b0a33b1d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:38:54.572586 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8340c8c1-f0f0-4c7f-93c1-66fdad0470c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:54.575762 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8340c8c1-f0f0-4c7f-93c1-66fdad0470c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:38:54.577932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1158/2327] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:38:54 2022] DELETE /identity/v3/projects/adc5f1c72d7a4bc3ad31ec8b0a33b1d4 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:39:04.178810 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=72,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:39:04.568005 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=58,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:39:08.324654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f583e5-c10b-4704-889f-f44e0356fdba None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:08.325820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f583e5-c10b-4704-889f-f44e0356fdba None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:08.326114 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f583e5-c10b-4704-889f-f44e0356fdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:08.346857 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-59f583e5-c10b-4704-889f-f44e0356fdba None None] MFA Rules not processed for user `f523b529906f4fa989d0e836d6a08944`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:39:08.349142 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-59f583e5-c10b-4704-889f-f44e0356fdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:08.412755 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1170/2328] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:39:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:39:09.700858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340c8c1-f0f0-4c7f-93c1-66fdad0470c3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:09.702337 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8340c8c1-f0f0-4c7f-93c1-66fdad0470c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:09.801515 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340c8c1-f0f0-4c7f-93c1-66fdad0470c3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:09.806641 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ffd72-bf8a-40c0-af09-39c4ade44b5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101864afd59543e699189917391e6ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:09.807376 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ffd72-bf8a-40c0-af09-39c4ade44b5c service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:09.807692 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ffd72-bf8a-40c0-af09-39c4ade44b5c service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:09.808001 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ffd72-bf8a-40c0-af09-39c4ade44b5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:09.810473 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-043ffd72-bf8a-40c0-af09-39c4ade44b5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:09.821587 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ffd72-bf8a-40c0-af09-39c4ade44b5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:09.823186 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ffd72-bf8a-40c0-af09-39c4ade44b5c service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:09.834279 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1159/2329] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:39:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:39:10.997225 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f583e5-c10b-4704-889f-f44e0356fdba None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:10.998781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-59f583e5-c10b-4704-889f-f44e0356fdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:11.078029 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f583e5-c10b-4704-889f-f44e0356fdba None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:11.082740 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e617782-ad98-4237-9489-c3a9bcdffdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:11.083683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e617782-ad98-4237-9489-c3a9bcdffdb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:11.083968 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e617782-ad98-4237-9489-c3a9bcdffdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:11.084244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e617782-ad98-4237-9489-c3a9bcdffdb5 admin admin] PATH_INFO: `/v3/users/b7b91cca223547f99f994f5267d6ac10` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:11.085621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e617782-ad98-4237-9489-c3a9bcdffdb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7b91cca223547f99f994f5267d6ac10)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:11.087673 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e617782-ad98-4237-9489-c3a9bcdffdb5 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:11.142480 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0e617782-ad98-4237-9489-c3a9bcdffdb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7b91cca223547f99f994f5267d6ac10'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:11.156378 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0e617782-ad98-4237-9489-c3a9bcdffdb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7b91cca223547f99f994f5267d6ac10'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:11.164251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0e617782-ad98-4237-9489-c3a9bcdffdb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7b91cca223547f99f994f5267d6ac10'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:11.172875 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0e617782-ad98-4237-9489-c3a9bcdffdb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7b91cca223547f99f994f5267d6ac10'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:11.173561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0e617782-ad98-4237-9489-c3a9bcdffdb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7b91cca223547f99f994f5267d6ac10'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:11.179835 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1171/2330] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:39:10 2022] DELETE /identity/v3/users/b7b91cca223547f99f994f5267d6ac10 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:39:11.184703 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d4f11-5405-4158-b7a0-1cecab5b1e79 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:11.185227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d4f11-5405-4158-b7a0-1cecab5b1e79 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:11.185334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d4f11-5405-4158-b7a0-1cecab5b1e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:11.205375 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-aa3d4f11-5405-4158-b7a0-1cecab5b1e79 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:39:11.206486 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-aa3d4f11-5405-4158-b7a0-1cecab5b1e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:11.285472 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1160/2331] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:39:11.295065 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e617782-ad98-4237-9489-c3a9bcdffdb5 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:11.297166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-0e617782-ad98-4237-9489-c3a9bcdffdb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:11.342909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:39:11.405148 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e617782-ad98-4237-9489-c3a9bcdffdb5 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:11.409197 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434beeb-537c-4e35-b4c7-ba8bec259216 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:11.409993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434beeb-537c-4e35-b4c7-ba8bec259216 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:11.410380 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434beeb-537c-4e35-b4c7-ba8bec259216 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:11.411434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434beeb-537c-4e35-b4c7-ba8bec259216 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:11.423544 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434beeb-537c-4e35-b4c7-ba8bec259216 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:11.425318 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434beeb-537c-4e35-b4c7-ba8bec259216 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:11.437802 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1172/2332] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:39:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:39:11.747748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d4f11-5405-4158-b7a0-1cecab5b1e79 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:11.749598 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-aa3d4f11-5405-4158-b7a0-1cecab5b1e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:11.770424 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d4f11-5405-4158-b7a0-1cecab5b1e79 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:11.777368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f9f69-71ba-4b5f-8c03-f768ad6627be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:11.778356 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f9f69-71ba-4b5f-8c03-f768ad6627be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:11.783467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f9f69-71ba-4b5f-8c03-f768ad6627be admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:11.785503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f9f69-71ba-4b5f-8c03-f768ad6627be admin admin] PATH_INFO: `/v3/projects/4dca85bb08ba42c5a91c02d1e5171585` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:11.787733 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f9f69-71ba-4b5f-8c03-f768ad6627be admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dca85bb08ba42c5a91c02d1e5171585)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:11.790563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f9f69-71ba-4b5f-8c03-f768ad6627be admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:11.850836 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6d4f9f69-71ba-4b5f-8c03-f768ad6627be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dca85bb08ba42c5a91c02d1e5171585', 'request_id': 'req-6d4f9f69-71ba-4b5f-8c03-f768ad6627be'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:11.860191 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6d4f9f69-71ba-4b5f-8c03-f768ad6627be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dca85bb08ba42c5a91c02d1e5171585', 'request_id': 'req-6d4f9f69-71ba-4b5f-8c03-f768ad6627be'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:11.867883 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6d4f9f69-71ba-4b5f-8c03-f768ad6627be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dca85bb08ba42c5a91c02d1e5171585', 'request_id': 'req-6d4f9f69-71ba-4b5f-8c03-f768ad6627be'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:11.869359 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:39:11.892932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6d4f9f69-71ba-4b5f-8c03-f768ad6627be admin admin] The token cache is being invalidate because project 4dca85bb08ba42c5a91c02d1e5171585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:39:11.893261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6d4f9f69-71ba-4b5f-8c03-f768ad6627be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:11.893938 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-6d4f9f69-71ba-4b5f-8c03-f768ad6627be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:11.895555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1161/2333] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:39:11 2022] DELETE /identity/v3/projects/4dca85bb08ba42c5a91c02d1e5171585 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:39:12.917603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434beeb-537c-4e35-b4c7-ba8bec259216 service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:12.919423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7434beeb-537c-4e35-b4c7-ba8bec259216 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:12.936001 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434beeb-537c-4e35-b4c7-ba8bec259216 service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:12.994684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c8cac-d85d-4950-a0ea-5619729065b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:12.995600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c8cac-d85d-4950-a0ea-5619729065b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:12.995959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c8cac-d85d-4950-a0ea-5619729065b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:12.996250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c8cac-d85d-4950-a0ea-5619729065b6 admin admin] PATH_INFO: `/v3/users/6de25a66435b4095b76d8f603a7a2759` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:12.997546 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c8cac-d85d-4950-a0ea-5619729065b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6de25a66435b4095b76d8f603a7a2759)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:12.999572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c8cac-d85d-4950-a0ea-5619729065b6 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:13.050226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-218c8cac-d85d-4950-a0ea-5619729065b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6de25a66435b4095b76d8f603a7a2759'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:13.063640 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-218c8cac-d85d-4950-a0ea-5619729065b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6de25a66435b4095b76d8f603a7a2759'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:13.072972 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-218c8cac-d85d-4950-a0ea-5619729065b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6de25a66435b4095b76d8f603a7a2759'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:13.081999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-218c8cac-d85d-4950-a0ea-5619729065b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6de25a66435b4095b76d8f603a7a2759'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:13.082680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-218c8cac-d85d-4950-a0ea-5619729065b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6de25a66435b4095b76d8f603a7a2759'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:13.088740 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1173/2334] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:39:12 2022] DELETE /identity/v3/users/6de25a66435b4095b76d8f603a7a2759 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:39:13.093946 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec94810-e5e0-4d20-b495-15ee6b83be89 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:13.094437 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec94810-e5e0-4d20-b495-15ee6b83be89 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:13.094555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec94810-e5e0-4d20-b495-15ee6b83be89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:13.115086 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5ec94810-e5e0-4d20-b495-15ee6b83be89 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:39:13.116312 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5ec94810-e5e0-4d20-b495-15ee6b83be89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:13.184486 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1162/2335] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:39:13.194827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c8cac-d85d-4950-a0ea-5619729065b6 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:13.196255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-218c8cac-d85d-4950-a0ea-5619729065b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:13.250464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:39:13.256942 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c8cac-d85d-4950-a0ea-5619729065b6 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:13.262087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b045fbe-3764-4d47-9077-f426f051db19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:13.262860 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b045fbe-3764-4d47-9077-f426f051db19 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:13.263172 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b045fbe-3764-4d47-9077-f426f051db19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:13.263452 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b045fbe-3764-4d47-9077-f426f051db19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:13.273173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b045fbe-3764-4d47-9077-f426f051db19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:13.274617 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b045fbe-3764-4d47-9077-f426f051db19 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:13.285146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1174/2336] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:39:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:39:13.686403 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec94810-e5e0-4d20-b495-15ee6b83be89 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:13.688155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5ec94810-e5e0-4d20-b495-15ee6b83be89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:13.703323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec94810-e5e0-4d20-b495-15ee6b83be89 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:13.708509 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07692c-01ce-48fc-a31d-ed1c660da011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:13.710220 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07692c-01ce-48fc-a31d-ed1c660da011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:13.710592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07692c-01ce-48fc-a31d-ed1c660da011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:13.711056 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07692c-01ce-48fc-a31d-ed1c660da011 admin admin] PATH_INFO: `/v3/projects/d445ce0afdb6479f904809eb5787cd2f` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:13.712652 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07692c-01ce-48fc-a31d-ed1c660da011 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d445ce0afdb6479f904809eb5787cd2f)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:13.714904 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07692c-01ce-48fc-a31d-ed1c660da011 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:13.763388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cd07692c-01ce-48fc-a31d-ed1c660da011 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd445ce0afdb6479f904809eb5787cd2f', 'request_id': 'req-cd07692c-01ce-48fc-a31d-ed1c660da011'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:13.770548 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cd07692c-01ce-48fc-a31d-ed1c660da011 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd445ce0afdb6479f904809eb5787cd2f', 'request_id': 'req-cd07692c-01ce-48fc-a31d-ed1c660da011'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:13.776128 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cd07692c-01ce-48fc-a31d-ed1c660da011 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd445ce0afdb6479f904809eb5787cd2f', 'request_id': 'req-cd07692c-01ce-48fc-a31d-ed1c660da011'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:13.787670 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cd07692c-01ce-48fc-a31d-ed1c660da011 admin admin] The token cache is being invalidate because project d445ce0afdb6479f904809eb5787cd2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:39:13.787963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cd07692c-01ce-48fc-a31d-ed1c660da011 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:13.788599 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cd07692c-01ce-48fc-a31d-ed1c660da011 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:39:13.790135 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1163/2337] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:39:13 2022] DELETE /identity/v3/projects/d445ce0afdb6479f904809eb5787cd2f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:39:13.795978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4629b0e-d334-41ba-b8b9-3fe21cdeb6f6 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:13.796274 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4629b0e-d334-41ba-b8b9-3fe21cdeb6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:13.796559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4629b0e-d334-41ba-b8b9-3fe21cdeb6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:13.814229 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-e4629b0e-d334-41ba-b8b9-3fe21cdeb6f6 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:39:13.815470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e4629b0e-d334-41ba-b8b9-3fe21cdeb6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:13.872541 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1175/2338] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:39:13.880419 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e496cc7-21b3-4f5e-bd2d-39e0c257e114 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:13.880719 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e496cc7-21b3-4f5e-bd2d-39e0c257e114 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:13.881009 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e496cc7-21b3-4f5e-bd2d-39e0c257e114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:13.899095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6e496cc7-21b3-4f5e-bd2d-39e0c257e114 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:39:13.900307 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6e496cc7-21b3-4f5e-bd2d-39e0c257e114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:13.908247 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1164/2339] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:39:13.912610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4629b0e-d334-41ba-b8b9-3fe21cdeb6f6 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:13.922974 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4629b0e-d334-41ba-b8b9-3fe21cdeb6f6 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:13.929283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badba1a8-1129-4402-8a53-3840807cab2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:13.930014 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badba1a8-1129-4402-8a53-3840807cab2c admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:13.930311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badba1a8-1129-4402-8a53-3840807cab2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:13.930644 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badba1a8-1129-4402-8a53-3840807cab2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:13.931458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badba1a8-1129-4402-8a53-3840807cab2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:39:13.931768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badba1a8-1129-4402-8a53-3840807cab2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:13.933961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badba1a8-1129-4402-8a53-3840807cab2c admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:13.946003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1176/2340] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:39:13.950829 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e496cc7-21b3-4f5e-bd2d-39e0c257e114 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:13.960042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e496cc7-21b3-4f5e-bd2d-39e0c257e114 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:13.965257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863665c-999e-4dc8-a769-d8d60a857607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:13.965913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863665c-999e-4dc8-a769-d8d60a857607 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:13.966206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863665c-999e-4dc8-a769-d8d60a857607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:13.966489 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863665c-999e-4dc8-a769-d8d60a857607 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:13.967215 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863665c-999e-4dc8-a769-d8d60a857607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:13.969112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863665c-999e-4dc8-a769-d8d60a857607 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:13.995048 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1165/2341] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:39:13 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:39:13.999310 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badba1a8-1129-4402-8a53-3840807cab2c admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:14.009520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badba1a8-1129-4402-8a53-3840807cab2c admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:14.066735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d264b6-045d-4103-bce4-5920b98af3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:14.067446 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d264b6-045d-4103-bce4-5920b98af3ab admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:14.067732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d264b6-045d-4103-bce4-5920b98af3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:14.068012 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d264b6-045d-4103-bce4-5920b98af3ab admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:14.068697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d264b6-045d-4103-bce4-5920b98af3ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:14.070599 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d264b6-045d-4103-bce4-5920b98af3ab admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:14.093968 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1177/2342] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:39:13 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:39:14.095218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863665c-999e-4dc8-a769-d8d60a857607 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:14.105933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863665c-999e-4dc8-a769-d8d60a857607 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:14.110614 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef62d1-ac03-4f8d-8556-7a5813f90590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:14.111311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef62d1-ac03-4f8d-8556-7a5813f90590 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:14.111599 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef62d1-ac03-4f8d-8556-7a5813f90590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:14.111895 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef62d1-ac03-4f8d-8556-7a5813f90590 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:14.112603 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef62d1-ac03-4f8d-8556-7a5813f90590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:14.114865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef62d1-ac03-4f8d-8556-7a5813f90590 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:14.132907 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1166/2343] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:39:14.137658 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d264b6-045d-4103-bce4-5920b98af3ab admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:14.147909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d264b6-045d-4103-bce4-5920b98af3ab admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:14.153273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1c5c5-3ae3-43c6-96c6-2b21eadc08e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:14.153971 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1c5c5-3ae3-43c6-96c6-2b21eadc08e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:14.154255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1c5c5-3ae3-43c6-96c6-2b21eadc08e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:14.154533 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1c5c5-3ae3-43c6-96c6-2b21eadc08e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:14.155353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f1c5c5-3ae3-43c6-96c6-2b21eadc08e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:14.156979 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f1c5c5-3ae3-43c6-96c6-2b21eadc08e3 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:14.167323 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1178/2344] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:39:14.171994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef62d1-ac03-4f8d-8556-7a5813f90590 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:14.182373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef62d1-ac03-4f8d-8556-7a5813f90590 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:14.187162 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a02b3-dd22-44f1-8b54-4a1361af20a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:14.188050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a02b3-dd22-44f1-8b54-4a1361af20a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:14.188340 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a02b3-dd22-44f1-8b54-4a1361af20a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:14.188626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a02b3-dd22-44f1-8b54-4a1361af20a1 admin admin] PATH_INFO: `/v3/projects/dceb245422db46eaa1440c39b5755aca/users/35838f82371148dda993ca515a1be79c/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:14.204608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a02b3-dd22-44f1-8b54-4a1361af20a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dceb245422db46eaa1440c39b5755aca, user_id=35838f82371148dda993ca515a1be79c, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:14.207210 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a02b3-dd22-44f1-8b54-4a1361af20a1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:14.222155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1167/2345] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:39:14 2022] PUT /identity/v3/projects/dceb245422db46eaa1440c39b5755aca/users/35838f82371148dda993ca515a1be79c/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:39:14.227356 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d158c6-f34f-4a3d-ab7b-52673c45f4f1 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:14.227814 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d158c6-f34f-4a3d-ab7b-52673c45f4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:14.228274 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d158c6-f34f-4a3d-ab7b-52673c45f4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:14.246989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-f9d158c6-f34f-4a3d-ab7b-52673c45f4f1 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:39:14.248193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-f9d158c6-f34f-4a3d-ab7b-52673c45f4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:14.295169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:39:14.307184 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1179/2346] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:39:14.316649 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a02b3-dd22-44f1-8b54-4a1361af20a1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:14.318374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-490a02b3-dd22-44f1-8b54-4a1361af20a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:14.333726 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:39:14.384735 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a02b3-dd22-44f1-8b54-4a1361af20a1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:14.390464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71445a3c-2a80-40ca-8181-5541c96368c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:14.391157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71445a3c-2a80-40ca-8181-5541c96368c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:14.391442 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71445a3c-2a80-40ca-8181-5541c96368c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:14.391720 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71445a3c-2a80-40ca-8181-5541c96368c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:14.401339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71445a3c-2a80-40ca-8181-5541c96368c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:14.403569 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71445a3c-2a80-40ca-8181-5541c96368c5 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:14.412987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1168/2347] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:39:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:39:18.589250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd38c4f-468d-4e4e-9903-3fbfcf91f7ed None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:18.590316 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd38c4f-468d-4e4e-9903-3fbfcf91f7ed None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:18.590601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd38c4f-468d-4e4e-9903-3fbfcf91f7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:18.618259 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-7cd38c4f-468d-4e4e-9903-3fbfcf91f7ed None None] MFA Rules not processed for user `35838f82371148dda993ca515a1be79c`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:39:18.619508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7cd38c4f-468d-4e4e-9903-3fbfcf91f7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:18.691781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1180/2348] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:39:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:39:18.696338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71445a3c-2a80-40ca-8181-5541c96368c5 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:18.708288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71445a3c-2a80-40ca-8181-5541c96368c5 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:18.712977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14d5b3-7d6a-421e-9f38-28c047ed5b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:18.713650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14d5b3-7d6a-421e-9f38-28c047ed5b67 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:18.713939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14d5b3-7d6a-421e-9f38-28c047ed5b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:18.714224 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14d5b3-7d6a-421e-9f38-28c047ed5b67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:18.715868 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14d5b3-7d6a-421e-9f38-28c047ed5b67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:18.718152 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14d5b3-7d6a-421e-9f38-28c047ed5b67 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:18.745355 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1169/2349] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:39:18 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:39:18.749416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd38c4f-468d-4e4e-9903-3fbfcf91f7ed None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:18.760709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd38c4f-468d-4e4e-9903-3fbfcf91f7ed None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:18.822108 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97feecd6-1197-4ad9-b28c-9aaa99ff95c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:18.822797 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97feecd6-1197-4ad9-b28c-9aaa99ff95c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:18.823114 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97feecd6-1197-4ad9-b28c-9aaa99ff95c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:18.823394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97feecd6-1197-4ad9-b28c-9aaa99ff95c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:18.824665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97feecd6-1197-4ad9-b28c-9aaa99ff95c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:18.826605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97feecd6-1197-4ad9-b28c-9aaa99ff95c2 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:18.850610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1181/2350] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:39:18 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:39:18.851878 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14d5b3-7d6a-421e-9f38-28c047ed5b67 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:18.875929 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14d5b3-7d6a-421e-9f38-28c047ed5b67 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:18.881230 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35693f30-3232-459e-b60f-761f72d3cfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:18.881915 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35693f30-3232-459e-b60f-761f72d3cfce admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:18.882199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35693f30-3232-459e-b60f-761f72d3cfce admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:18.883682 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35693f30-3232-459e-b60f-761f72d3cfce admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:18.883682 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35693f30-3232-459e-b60f-761f72d3cfce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:18.885930 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35693f30-3232-459e-b60f-761f72d3cfce admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:18.898011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1170/2351] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:39:18.902206 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97feecd6-1197-4ad9-b28c-9aaa99ff95c2 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:18.912722 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97feecd6-1197-4ad9-b28c-9aaa99ff95c2 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:18.917527 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ff0b1-829e-4b18-9eac-5c3ca252d0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:18.919221 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ff0b1-829e-4b18-9eac-5c3ca252d0a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:18.919506 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ff0b1-829e-4b18-9eac-5c3ca252d0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:18.919834 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ff0b1-829e-4b18-9eac-5c3ca252d0a6 admin admin] PATH_INFO: `/v3/projects/6bb38a54c255484b8bd9f316ae754708/users/9ad72906339a45edbadce7983c5679ab/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:18.944226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ff0b1-829e-4b18-9eac-5c3ca252d0a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb38a54c255484b8bd9f316ae754708, user_id=9ad72906339a45edbadce7983c5679ab, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:18.946853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ff0b1-829e-4b18-9eac-5c3ca252d0a6 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:18.961600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1182/2352] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:39:18 2022] PUT /identity/v3/projects/6bb38a54c255484b8bd9f316ae754708/users/9ad72906339a45edbadce7983c5679ab/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:39:18.965854 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35693f30-3232-459e-b60f-761f72d3cfce admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:18.975955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35693f30-3232-459e-b60f-761f72d3cfce admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:19.037694 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e24c6e-c88a-46f6-9a09-d787cd1855ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:19.038447 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e24c6e-c88a-46f6-9a09-d787cd1855ee admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:19.038738 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e24c6e-c88a-46f6-9a09-d787cd1855ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:19.039069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e24c6e-c88a-46f6-9a09-d787cd1855ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:19.039803 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e24c6e-c88a-46f6-9a09-d787cd1855ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:19.041411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e24c6e-c88a-46f6-9a09-d787cd1855ee admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:19.052523 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1171/2353] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:39:19.056811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ff0b1-829e-4b18-9eac-5c3ca252d0a6 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:19.068491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ff0b1-829e-4b18-9eac-5c3ca252d0a6 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:19.073301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445a8e2-88a4-4340-9825-b28e3fe05ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:19.074232 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445a8e2-88a4-4340-9825-b28e3fe05ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:19.074528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445a8e2-88a4-4340-9825-b28e3fe05ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:19.074846 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445a8e2-88a4-4340-9825-b28e3fe05ccb admin admin] PATH_INFO: `/v3/projects/6bb38a54c255484b8bd9f316ae754708/users/9ad72906339a45edbadce7983c5679ab/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:19.076929 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445a8e2-88a4-4340-9825-b28e3fe05ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb38a54c255484b8bd9f316ae754708, user_id=9ad72906339a45edbadce7983c5679ab, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:19.079648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d445a8e2-88a4-4340-9825-b28e3fe05ccb admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:19.090758 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1183/2354] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:39:19 2022] PUT /identity/v3/projects/6bb38a54c255484b8bd9f316ae754708/users/9ad72906339a45edbadce7983c5679ab/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:39:23.744395 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7bc67-fb31-423a-9639-3fedeefe0b6b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:23.744883 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7bc67-fb31-423a-9639-3fedeefe0b6b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:23.744997 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7bc67-fb31-423a-9639-3fedeefe0b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:23.776155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-d0d7bc67-fb31-423a-9639-3fedeefe0b6b None None] MFA Rules not processed for user `9ad72906339a45edbadce7983c5679ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:39:23.777451 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d0d7bc67-fb31-423a-9639-3fedeefe0b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:23.851905 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1172/2355] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:39:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:39:23.866009 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445a8e2-88a4-4340-9825-b28e3fe05ccb admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:23.867889 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d445a8e2-88a4-4340-9825-b28e3fe05ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:23.919120 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:39:23.982631 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d445a8e2-88a4-4340-9825-b28e3fe05ccb admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:23.989574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1f89f-6e4f-43a3-a7cb-32260fbcba4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:23.990242 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1f89f-6e4f-43a3-a7cb-32260fbcba4e service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:23.990539 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1f89f-6e4f-43a3-a7cb-32260fbcba4e service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:23.990824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1f89f-6e4f-43a3-a7cb-32260fbcba4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:24.050983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1f89f-6e4f-43a3-a7cb-32260fbcba4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:24.052590 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1f89f-6e4f-43a3-a7cb-32260fbcba4e service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:24.063047 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1184/2356] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:39:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:39:25.291936 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3c203-4ee0-4099-9f80-49062faef199 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:25.292484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3c203-4ee0-4099-9f80-49062faef199 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:25.292484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3c203-4ee0-4099-9f80-49062faef199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:25.299687 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:39:25.312621 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-80a3c203-4ee0-4099-9f80-49062faef199 None None] MFA Rules not processed for user `35838f82371148dda993ca515a1be79c`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:39:25.313858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-80a3c203-4ee0-4099-9f80-49062faef199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:25.322865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1173/2357] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:39:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:39:27.467066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1f89f-6e4f-43a3-a7cb-32260fbcba4e service nova] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:39:27.468666 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-09c1f89f-6e4f-43a3-a7cb-32260fbcba4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:27.551110 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1f89f-6e4f-43a3-a7cb-32260fbcba4e service nova] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:39:27.564407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ea9d3-26fa-43e8-b13a-166eaafd5683 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:39:27.565190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ea9d3-26fa-43e8-b13a-166eaafd5683 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:39:27.565595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ea9d3-26fa-43e8-b13a-166eaafd5683 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:39:27.565955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ea9d3-26fa-43e8-b13a-166eaafd5683 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:39:27.568250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e28ea9d3-26fa-43e8-b13a-166eaafd5683 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:39:27.676937 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ea9d3-26fa-43e8-b13a-166eaafd5683 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:39:27.678520 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ea9d3-26fa-43e8-b13a-166eaafd5683 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:39:27.683209 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:39:27.691662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1185/2358] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:39:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:39:35.310210 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:39:37.686288 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:40:01.593570 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3c203-4ee0-4099-9f80-49062faef199 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:40:01.603179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3c203-4ee0-4099-9f80-49062faef199 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:40:01.606729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67243795-fa8e-4e59-9a96-c41fe3ac5cff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:40:01.607436 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67243795-fa8e-4e59-9a96-c41fe3ac5cff service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:40:01.607726 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67243795-fa8e-4e59-9a96-c41fe3ac5cff service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:40:01.608005 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67243795-fa8e-4e59-9a96-c41fe3ac5cff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:40:01.617734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67243795-fa8e-4e59-9a96-c41fe3ac5cff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:40:01.619195 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67243795-fa8e-4e59-9a96-c41fe3ac5cff service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:40:01.629354 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1174/2359] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:40:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:40:09.420881 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc586899-06f4-4f5d-81fd-dd1d754f1a7c None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:40:09.421436 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc586899-06f4-4f5d-81fd-dd1d754f1a7c None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:40:09.421436 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc586899-06f4-4f5d-81fd-dd1d754f1a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:40:09.441761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-fc586899-06f4-4f5d-81fd-dd1d754f1a7c None None] MFA Rules not processed for user `03486c7630314392b85271455d12aba2`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:40:09.443097 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-fc586899-06f4-4f5d-81fd-dd1d754f1a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:40:09.501462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1186/2360] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:40:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:40:11.624357 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:40:19.494618 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=25 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:40:27.536423 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2a6c8-1bf5-4a3d-b2dd-85305e5ca372 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:40:27.536944 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2a6c8-1bf5-4a3d-b2dd-85305e5ca372 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:40:27.537062 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2a6c8-1bf5-4a3d-b2dd-85305e5ca372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:40:27.554787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-34b2a6c8-1bf5-4a3d-b2dd-85305e5ca372 None None] MFA Rules not processed for user `dc246e144afc46698000f7f53ef6489e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:40:27.556046 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-34b2a6c8-1bf5-4a3d-b2dd-85305e5ca372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:40:27.610545 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1175/2361] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:40:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:40:37.603954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=25 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:40:57.336293 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc586899-06f4-4f5d-81fd-dd1d754f1a7c None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:40:57.346439 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc586899-06f4-4f5d-81fd-dd1d754f1a7c None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:40:57.350039 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bd350-8144-460b-9d63-1201a98c830b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:40:57.351251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bd350-8144-460b-9d63-1201a98c830b service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:40:57.351645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bd350-8144-460b-9d63-1201a98c830b service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:40:57.351935 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bd350-8144-460b-9d63-1201a98c830b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:40:57.353799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2e9bd350-8144-460b-9d63-1201a98c830b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:40:57.367957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9bd350-8144-460b-9d63-1201a98c830b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:40:57.369387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9bd350-8144-460b-9d63-1201a98c830b service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:40:57.380568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1187/2362] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:40:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:40:58.515504 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2a6c8-1bf5-4a3d-b2dd-85305e5ca372 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:40:58.529575 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2a6c8-1bf5-4a3d-b2dd-85305e5ca372 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:40:58.535119 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fefad5-d9b1-4d12-8908-b98f4faddf4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:40:58.535778 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fefad5-d9b1-4d12-8908-b98f4faddf4a service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:40:58.536066 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fefad5-d9b1-4d12-8908-b98f4faddf4a service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:40:58.536350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fefad5-d9b1-4d12-8908-b98f4faddf4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:40:58.538979 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-57fefad5-d9b1-4d12-8908-b98f4faddf4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:40:58.598380 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fefad5-d9b1-4d12-8908-b98f4faddf4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:40:58.600019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fefad5-d9b1-4d12-8908-b98f4faddf4a service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:40:58.609904 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1176/2363] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:40:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:40:58.661117 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bd350-8144-460b-9d63-1201a98c830b service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:40:58.670664 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bd350-8144-460b-9d63-1201a98c830b service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:40:58.674321 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b739ff6-facd-4146-96d0-ddf873f3414d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:40:58.674998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b739ff6-facd-4146-96d0-ddf873f3414d service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:40:58.675288 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b739ff6-facd-4146-96d0-ddf873f3414d service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:40:58.675569 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b739ff6-facd-4146-96d0-ddf873f3414d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:40:58.677350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-4b739ff6-facd-4146-96d0-ddf873f3414d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:40:58.690521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b739ff6-facd-4146-96d0-ddf873f3414d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:40:58.691996 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b739ff6-facd-4146-96d0-ddf873f3414d service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:40:58.703463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1188/2364] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:40:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:41:04.285325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df7328-6237-40cb-be16-9e59c3e006f6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:04.285844 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df7328-6237-40cb-be16-9e59c3e006f6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:04.285961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df7328-6237-40cb-be16-9e59c3e006f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:04.304332 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e4df7328-6237-40cb-be16-9e59c3e006f6 None None] MFA Rules not processed for user `f53284773f7a430db3d0e16d3c2cd262`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:41:04.305595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e4df7328-6237-40cb-be16-9e59c3e006f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:04.312444 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1177/2365] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:41:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:41:08.698748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=12 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:41:14.298506 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=31 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:41:22.269733 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b739ff6-facd-4146-96d0-ddf873f3414d service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:22.271252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-4b739ff6-facd-4146-96d0-ddf873f3414d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:22.284653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b739ff6-facd-4146-96d0-ddf873f3414d service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:22.290328 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb52bc4-2219-43e3-b692-3c1db7d54857 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:22.290997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb52bc4-2219-43e3-b692-3c1db7d54857 service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:22.291283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb52bc4-2219-43e3-b692-3c1db7d54857 service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:22.291560 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb52bc4-2219-43e3-b692-3c1db7d54857 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:22.293272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-4eb52bc4-2219-43e3-b692-3c1db7d54857 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:22.303288 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb52bc4-2219-43e3-b692-3c1db7d54857 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:22.304856 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb52bc4-2219-43e3-b692-3c1db7d54857 service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:22.314212 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1189/2366] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:41:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:41:22.483202 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df7328-6237-40cb-be16-9e59c3e006f6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:22.484644 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e4df7328-6237-40cb-be16-9e59c3e006f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:22.580901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df7328-6237-40cb-be16-9e59c3e006f6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:22.585049 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96295952-5b3e-4a0d-b573-580184b4de3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:22.585709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96295952-5b3e-4a0d-b573-580184b4de3e service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:22.586000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96295952-5b3e-4a0d-b573-580184b4de3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:22.586322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96295952-5b3e-4a0d-b573-580184b4de3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:22.588752 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-96295952-5b3e-4a0d-b573-580184b4de3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:22.605040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96295952-5b3e-4a0d-b573-580184b4de3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:22.606579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96295952-5b3e-4a0d-b573-580184b4de3e service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:22.630412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1178/2367] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:41:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:41:30.520834 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb52bc4-2219-43e3-b692-3c1db7d54857 service nova] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:30.532643 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb52bc4-2219-43e3-b692-3c1db7d54857 service nova] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:30.539346 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84eaa27-8b32-4232-9453-a4d902c9af1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:30.539949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84eaa27-8b32-4232-9453-a4d902c9af1d service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:30.540232 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84eaa27-8b32-4232-9453-a4d902c9af1d service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:30.540511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84eaa27-8b32-4232-9453-a4d902c9af1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:30.543080 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b84eaa27-8b32-4232-9453-a4d902c9af1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:30.555994 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84eaa27-8b32-4232-9453-a4d902c9af1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:30.559236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84eaa27-8b32-4232-9453-a4d902c9af1d service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:30.585973 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1190/2368] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:41:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:41:32.619471 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=35 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:41:40.567448 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=12 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:41:42.398327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa3787-6639-46cd-baa0-8ddd55c85b7c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:42.398770 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa3787-6639-46cd-baa0-8ddd55c85b7c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:42.398895 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa3787-6639-46cd-baa0-8ddd55c85b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:42.417679 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-efaa3787-6639-46cd-baa0-8ddd55c85b7c None None] MFA Rules not processed for user `03486c7630314392b85271455d12aba2`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:41:42.418937 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-efaa3787-6639-46cd-baa0-8ddd55c85b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:42.425830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1179/2369] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:41:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:41:42.736256 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84eaa27-8b32-4232-9453-a4d902c9af1d service nova] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:42.747065 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84eaa27-8b32-4232-9453-a4d902c9af1d service nova] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:42.750998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267ef23-cde1-4a4c-9e5b-053ab8aa2f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:42.751692 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267ef23-cde1-4a4c-9e5b-053ab8aa2f1c service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:42.752016 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267ef23-cde1-4a4c-9e5b-053ab8aa2f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:42.752302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267ef23-cde1-4a4c-9e5b-053ab8aa2f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:42.754261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2267ef23-cde1-4a4c-9e5b-053ab8aa2f1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:42.765296 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267ef23-cde1-4a4c-9e5b-053ab8aa2f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:42.766761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267ef23-cde1-4a4c-9e5b-053ab8aa2f1c service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:42.774826 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1191/2370] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:41:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:41:43.107369 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa3787-6639-46cd-baa0-8ddd55c85b7c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:43.108816 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-efaa3787-6639-46cd-baa0-8ddd55c85b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:43.192286 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa3787-6639-46cd-baa0-8ddd55c85b7c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:43.197256 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b773d7e-aa47-4d35-98b3-9cb507ecfccd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101864afd59543e699189917391e6ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:43.198012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b773d7e-aa47-4d35-98b3-9cb507ecfccd service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:43.198298 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b773d7e-aa47-4d35-98b3-9cb507ecfccd service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:43.198719 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b773d7e-aa47-4d35-98b3-9cb507ecfccd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:43.200858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9b773d7e-aa47-4d35-98b3-9cb507ecfccd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:43.214730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b773d7e-aa47-4d35-98b3-9cb507ecfccd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:43.216400 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b773d7e-aa47-4d35-98b3-9cb507ecfccd service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:43.227008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1180/2371] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:41:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:41:45.924717 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267ef23-cde1-4a4c-9e5b-053ab8aa2f1c service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:45.935720 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267ef23-cde1-4a4c-9e5b-053ab8aa2f1c service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:45.941265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1d0b6-7f1f-468b-8f74-e2cb930e0da2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:45.941941 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1d0b6-7f1f-468b-8f74-e2cb930e0da2 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:45.942830 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1d0b6-7f1f-468b-8f74-e2cb930e0da2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:45.943152 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1d0b6-7f1f-468b-8f74-e2cb930e0da2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:45.944976 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d9c1d0b6-7f1f-468b-8f74-e2cb930e0da2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:45.956266 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1d0b6-7f1f-468b-8f74-e2cb930e0da2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:45.957723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1d0b6-7f1f-468b-8f74-e2cb930e0da2 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:45.967814 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1192/2372] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:41:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:41:46.069633 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b773d7e-aa47-4d35-98b3-9cb507ecfccd service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:46.081802 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b773d7e-aa47-4d35-98b3-9cb507ecfccd service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:46.085329 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2654c3-7349-4add-991f-ea31e54451f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:46.085987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2654c3-7349-4add-991f-ea31e54451f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:46.086265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2654c3-7349-4add-991f-ea31e54451f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:46.086729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2654c3-7349-4add-991f-ea31e54451f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:46.089096 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ad2654c3-7349-4add-991f-ea31e54451f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:46.163240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2654c3-7349-4add-991f-ea31e54451f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:46.165063 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2654c3-7349-4add-991f-ea31e54451f3 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:46.176749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1181/2373] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:41:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:41:49.750177 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1d0b6-7f1f-468b-8f74-e2cb930e0da2 service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:49.752055 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d9c1d0b6-7f1f-468b-8f74-e2cb930e0da2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:49.769225 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1d0b6-7f1f-468b-8f74-e2cb930e0da2 service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:49.775733 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892b453-7d26-4650-ab60-1a2165ce35d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101864afd59543e699189917391e6ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:49.776814 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892b453-7d26-4650-ab60-1a2165ce35d4 service placement] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:49.777336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892b453-7d26-4650-ab60-1a2165ce35d4 service placement] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:49.777838 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892b453-7d26-4650-ab60-1a2165ce35d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:49.779763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-6892b453-7d26-4650-ab60-1a2165ce35d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:49.793314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892b453-7d26-4650-ab60-1a2165ce35d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:49.795281 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892b453-7d26-4650-ab60-1a2165ce35d4 service placement] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:49.814496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2654c3-7349-4add-991f-ea31e54451f3 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:49.816116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1193/2374] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:41:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:41:49.816961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ad2654c3-7349-4add-991f-ea31e54451f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:49.832144 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2654c3-7349-4add-991f-ea31e54451f3 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:49.837233 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b59cc-0067-45ca-91a7-2ccfd4ebf3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:49.838988 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b59cc-0067-45ca-91a7-2ccfd4ebf3c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:49.839285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b59cc-0067-45ca-91a7-2ccfd4ebf3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:49.839568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b59cc-0067-45ca-91a7-2ccfd4ebf3c0 admin admin] PATH_INFO: `/v3/users/dc246e144afc46698000f7f53ef6489e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:49.843802 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b59cc-0067-45ca-91a7-2ccfd4ebf3c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc246e144afc46698000f7f53ef6489e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:49.845992 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b59cc-0067-45ca-91a7-2ccfd4ebf3c0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:49.893292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-563b59cc-0067-45ca-91a7-2ccfd4ebf3c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc246e144afc46698000f7f53ef6489e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:49.901075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-563b59cc-0067-45ca-91a7-2ccfd4ebf3c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc246e144afc46698000f7f53ef6489e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:49.926319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-563b59cc-0067-45ca-91a7-2ccfd4ebf3c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc246e144afc46698000f7f53ef6489e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:49.926947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-563b59cc-0067-45ca-91a7-2ccfd4ebf3c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc246e144afc46698000f7f53ef6489e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:49.927575 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-563b59cc-0067-45ca-91a7-2ccfd4ebf3c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc246e144afc46698000f7f53ef6489e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:49.934580 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:41:49.949993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1182/2375] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:41:49 2022] DELETE /identity/v3/users/dc246e144afc46698000f7f53ef6489e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:41:50.160457 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892b453-7d26-4650-ab60-1a2165ce35d4 service placement] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:50.173899 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892b453-7d26-4650-ab60-1a2165ce35d4 service placement] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:50.256577 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe53b51-2cf9-467a-80b0-141f616bee6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:50.257665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe53b51-2cf9-467a-80b0-141f616bee6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:50.257963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe53b51-2cf9-467a-80b0-141f616bee6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:50.258273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe53b51-2cf9-467a-80b0-141f616bee6c admin admin] PATH_INFO: `/v3/projects/192d1c8d15ce462dbf10fc3b519ab253` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:50.259718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe53b51-2cf9-467a-80b0-141f616bee6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=192d1c8d15ce462dbf10fc3b519ab253)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:50.261947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe53b51-2cf9-467a-80b0-141f616bee6c admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:50.311702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-8fe53b51-2cf9-467a-80b0-141f616bee6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '192d1c8d15ce462dbf10fc3b519ab253', 'request_id': 'req-8fe53b51-2cf9-467a-80b0-141f616bee6c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:50.317944 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-8fe53b51-2cf9-467a-80b0-141f616bee6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '192d1c8d15ce462dbf10fc3b519ab253', 'request_id': 'req-8fe53b51-2cf9-467a-80b0-141f616bee6c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:50.326496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-8fe53b51-2cf9-467a-80b0-141f616bee6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '192d1c8d15ce462dbf10fc3b519ab253', 'request_id': 'req-8fe53b51-2cf9-467a-80b0-141f616bee6c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:50.338077 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-8fe53b51-2cf9-467a-80b0-141f616bee6c admin admin] The token cache is being invalidate because project 192d1c8d15ce462dbf10fc3b519ab253 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:41:50.338372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-8fe53b51-2cf9-467a-80b0-141f616bee6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:50.339074 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-8fe53b51-2cf9-467a-80b0-141f616bee6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:41:50.340685 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1194/2376] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:41:50 2022] DELETE /identity/v3/projects/192d1c8d15ce462dbf10fc3b519ab253 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:41:50.345779 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f582924-646e-4895-980f-e2bbd5b0f978 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:50.346103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f582924-646e-4895-980f-e2bbd5b0f978 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:50.346396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f582924-646e-4895-980f-e2bbd5b0f978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:50.367604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7f582924-646e-4895-980f-e2bbd5b0f978 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:41:50.368977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7f582924-646e-4895-980f-e2bbd5b0f978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:50.431915 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1183/2377] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:41:50.439876 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10d01d-c03d-42e0-aa59-0f4098d2ad21 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:50.440307 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10d01d-c03d-42e0-aa59-0f4098d2ad21 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:50.440660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f10d01d-c03d-42e0-aa59-0f4098d2ad21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:50.460094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-9f10d01d-c03d-42e0-aa59-0f4098d2ad21 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:41:50.461392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-9f10d01d-c03d-42e0-aa59-0f4098d2ad21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:50.470271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1195/2378] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:41:50.474314 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f582924-646e-4895-980f-e2bbd5b0f978 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:50.484453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f582924-646e-4895-980f-e2bbd5b0f978 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:50.489061 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b7a8d-589a-4489-95f6-3f21c06cd1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:50.489721 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b7a8d-589a-4489-95f6-3f21c06cd1bd admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:50.490018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b7a8d-589a-4489-95f6-3f21c06cd1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:50.490304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b7a8d-589a-4489-95f6-3f21c06cd1bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:50.491061 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b7a8d-589a-4489-95f6-3f21c06cd1bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:41:50.491395 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b7a8d-589a-4489-95f6-3f21c06cd1bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:50.493053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b7a8d-589a-4489-95f6-3f21c06cd1bd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:50.505159 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1184/2379] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:41:50.509868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10d01d-c03d-42e0-aa59-0f4098d2ad21 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:50.519720 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f10d01d-c03d-42e0-aa59-0f4098d2ad21 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:50.524739 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444996f-4a70-4af3-b603-30bbd226d7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:50.526138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444996f-4a70-4af3-b603-30bbd226d7e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:50.526426 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444996f-4a70-4af3-b603-30bbd226d7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:50.526731 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444996f-4a70-4af3-b603-30bbd226d7e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:50.527565 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444996f-4a70-4af3-b603-30bbd226d7e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:50.529458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444996f-4a70-4af3-b603-30bbd226d7e5 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:50.555973 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1196/2380] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:41:50 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:41:50.560239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b7a8d-589a-4489-95f6-3f21c06cd1bd admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:50.570230 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b7a8d-589a-4489-95f6-3f21c06cd1bd admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:50.627546 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f854830-bbc2-4049-8759-e5d9f901fb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:50.628216 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f854830-bbc2-4049-8759-e5d9f901fb66 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:50.628501 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f854830-bbc2-4049-8759-e5d9f901fb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:50.628782 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f854830-bbc2-4049-8759-e5d9f901fb66 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:50.629481 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f854830-bbc2-4049-8759-e5d9f901fb66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:50.631390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f854830-bbc2-4049-8759-e5d9f901fb66 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:50.653157 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1185/2381] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:41:50 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:41:50.657199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444996f-4a70-4af3-b603-30bbd226d7e5 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:50.668146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444996f-4a70-4af3-b603-30bbd226d7e5 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:50.673007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52da3d9-9891-49c2-a920-22da15db5c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:50.673677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52da3d9-9891-49c2-a920-22da15db5c27 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:50.673969 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52da3d9-9891-49c2-a920-22da15db5c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:50.674255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52da3d9-9891-49c2-a920-22da15db5c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:50.675026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52da3d9-9891-49c2-a920-22da15db5c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:50.676635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52da3d9-9891-49c2-a920-22da15db5c27 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:50.687455 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1197/2382] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:41:50.691563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f854830-bbc2-4049-8759-e5d9f901fb66 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:50.702399 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f854830-bbc2-4049-8759-e5d9f901fb66 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:50.707079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3acf78-838a-4ca6-907d-c302cbd24c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:50.707744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3acf78-838a-4ca6-907d-c302cbd24c2b admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:50.708029 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3acf78-838a-4ca6-907d-c302cbd24c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:50.708310 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3acf78-838a-4ca6-907d-c302cbd24c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:50.709091 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3acf78-838a-4ca6-907d-c302cbd24c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:50.710740 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3acf78-838a-4ca6-907d-c302cbd24c2b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:50.720894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1186/2383] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:41:50.725246 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52da3d9-9891-49c2-a920-22da15db5c27 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:50.735900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52da3d9-9891-49c2-a920-22da15db5c27 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:50.740659 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299b426-7a13-45b4-8961-561e3886f164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:50.741551 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299b426-7a13-45b4-8961-561e3886f164 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:50.741846 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299b426-7a13-45b4-8961-561e3886f164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:50.742130 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299b426-7a13-45b4-8961-561e3886f164 admin admin] PATH_INFO: `/v3/projects/9ab3a2e6c9634a06b33f1d82a28dc3f1/users/1a93ea9f0a3f47d68e53c464c1d52c92/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:50.751046 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:41:50.761332 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299b426-7a13-45b4-8961-561e3886f164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ab3a2e6c9634a06b33f1d82a28dc3f1, user_id=1a93ea9f0a3f47d68e53c464c1d52c92, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:50.764589 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299b426-7a13-45b4-8961-561e3886f164 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:50.777692 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1198/2384] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:41:50 2022] PUT /identity/v3/projects/9ab3a2e6c9634a06b33f1d82a28dc3f1/users/1a93ea9f0a3f47d68e53c464c1d52c92/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:41:50.782829 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3b38f-98a9-468a-8e76-3b807351ee1d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:50.783160 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3b38f-98a9-468a-8e76-3b807351ee1d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:50.783444 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3b38f-98a9-468a-8e76-3b807351ee1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:50.801261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7bb3b38f-98a9-468a-8e76-3b807351ee1d None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:41:50.802460 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7bb3b38f-98a9-468a-8e76-3b807351ee1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:50.851524 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:41:50.865149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1187/2385] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:41:50.875305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299b426-7a13-45b4-8961-561e3886f164 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:50.877008 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3299b426-7a13-45b4-8961-561e3886f164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:50.933822 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299b426-7a13-45b4-8961-561e3886f164 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:50.938324 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4515ca2-e470-4bdb-b7f1-c8c415b9dfbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:50.939020 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4515ca2-e470-4bdb-b7f1-c8c415b9dfbb service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:50.939311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4515ca2-e470-4bdb-b7f1-c8c415b9dfbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:50.939597 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4515ca2-e470-4bdb-b7f1-c8c415b9dfbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:50.949315 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4515ca2-e470-4bdb-b7f1-c8c415b9dfbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:50.950778 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4515ca2-e470-4bdb-b7f1-c8c415b9dfbb service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:50.960299 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1199/2386] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:41:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:41:51.631520 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3b38f-98a9-468a-8e76-3b807351ee1d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:51.632871 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7bb3b38f-98a9-468a-8e76-3b807351ee1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:51.722649 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3b38f-98a9-468a-8e76-3b807351ee1d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:51.729586 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836b0ed-048d-40c3-be9b-fc2c0d6da9ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:51.730290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836b0ed-048d-40c3-be9b-fc2c0d6da9ea service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:51.730584 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836b0ed-048d-40c3-be9b-fc2c0d6da9ea service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:51.730871 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836b0ed-048d-40c3-be9b-fc2c0d6da9ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:51.732713 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6836b0ed-048d-40c3-be9b-fc2c0d6da9ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:51.745529 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836b0ed-048d-40c3-be9b-fc2c0d6da9ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:51.747151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836b0ed-048d-40c3-be9b-fc2c0d6da9ea service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:51.763996 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1188/2387] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:41:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:41:55.573100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bcc5a-10c9-4f78-ac90-76904b34500b None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:55.573654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bcc5a-10c9-4f78-ac90-76904b34500b None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:55.573654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bcc5a-10c9-4f78-ac90-76904b34500b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:55.605921 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-ee0bcc5a-10c9-4f78-ac90-76904b34500b None None] MFA Rules not processed for user `f523b529906f4fa989d0e836d6a08944`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:41:55.607274 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-ee0bcc5a-10c9-4f78-ac90-76904b34500b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:55.722703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1200/2388] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:41:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:41:55.965917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6812016-17aa-4099-9e2c-dc8876358469 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:55.966794 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6812016-17aa-4099-9e2c-dc8876358469 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:55.967266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6812016-17aa-4099-9e2c-dc8876358469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:55.998496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b6812016-17aa-4099-9e2c-dc8876358469 None None] MFA Rules not processed for user `1a93ea9f0a3f47d68e53c464c1d52c92`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:41:56.000042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b6812016-17aa-4099-9e2c-dc8876358469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:56.076907 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1189/2389] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:41:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:41:56.092002 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bcc5a-10c9-4f78-ac90-76904b34500b None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:56.107372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bcc5a-10c9-4f78-ac90-76904b34500b None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:56.114044 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84606bb2-9394-4e3a-852f-543c7c1d3448 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:56.114814 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84606bb2-9394-4e3a-852f-543c7c1d3448 service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:56.115199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84606bb2-9394-4e3a-852f-543c7c1d3448 service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:56.115557 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84606bb2-9394-4e3a-852f-543c7c1d3448 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:56.126342 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84606bb2-9394-4e3a-852f-543c7c1d3448 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:56.128495 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84606bb2-9394-4e3a-852f-543c7c1d3448 service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:56.144487 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1201/2390] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:41:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:41:56.366730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6812016-17aa-4099-9e2c-dc8876358469 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:41:56.372292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b6812016-17aa-4099-9e2c-dc8876358469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:56.392688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6812016-17aa-4099-9e2c-dc8876358469 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:41:56.396865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b111ab2-77ff-4eb8-a6d0-1ddb5aee60f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:41:56.397601 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b111ab2-77ff-4eb8-a6d0-1ddb5aee60f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:56.398101 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b111ab2-77ff-4eb8-a6d0-1ddb5aee60f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:56.398639 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b111ab2-77ff-4eb8-a6d0-1ddb5aee60f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:56.401582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4b111ab2-77ff-4eb8-a6d0-1ddb5aee60f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:56.430059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b111ab2-77ff-4eb8-a6d0-1ddb5aee60f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:41:56.438446 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b111ab2-77ff-4eb8-a6d0-1ddb5aee60f5 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:41:56.454040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1190/2391] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:41:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:41:57.916549 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3846a4-a430-45f5-bcbe-b69bde05f7ac None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:41:57.917463 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3846a4-a430-45f5-bcbe-b69bde05f7ac None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:41:57.918151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3846a4-a430-45f5-bcbe-b69bde05f7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:41:57.943065 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-ab3846a4-a430-45f5-bcbe-b69bde05f7ac None None] MFA Rules not processed for user `1a93ea9f0a3f47d68e53c464c1d52c92`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:41:57.946422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-ab3846a4-a430-45f5-bcbe-b69bde05f7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:41:57.957315 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1202/2392] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:41:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:06.445967 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=74 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:07.934549 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=65,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:37.911455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b111ab2-77ff-4eb8-a6d0-1ddb5aee60f5 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:37.921199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b111ab2-77ff-4eb8-a6d0-1ddb5aee60f5 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:37.924952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62962fa1-9913-4444-ab91-c03fd3d2ae10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:37.925636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62962fa1-9913-4444-ab91-c03fd3d2ae10 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:37.925926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62962fa1-9913-4444-ab91-c03fd3d2ae10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:37.926207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62962fa1-9913-4444-ab91-c03fd3d2ae10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:37.928052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-62962fa1-9913-4444-ab91-c03fd3d2ae10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:37.942430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62962fa1-9913-4444-ab91-c03fd3d2ae10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:37.944012 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62962fa1-9913-4444-ab91-c03fd3d2ae10 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:37.955127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1191/2393] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:42:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:42:41.205270 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3846a4-a430-45f5-bcbe-b69bde05f7ac None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:41.206742 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-ab3846a4-a430-45f5-bcbe-b69bde05f7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:41.224104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3846a4-a430-45f5-bcbe-b69bde05f7ac None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:41.228761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e5b4a-7053-44d3-b46e-8a5b05217e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:41.230619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e5b4a-7053-44d3-b46e-8a5b05217e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:41.231051 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e5b4a-7053-44d3-b46e-8a5b05217e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:41.231315 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e5b4a-7053-44d3-b46e-8a5b05217e46 admin admin] PATH_INFO: `/v3/users/f53284773f7a430db3d0e16d3c2cd262` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:41.232607 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e5b4a-7053-44d3-b46e-8a5b05217e46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f53284773f7a430db3d0e16d3c2cd262)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:41.234548 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e5b4a-7053-44d3-b46e-8a5b05217e46 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:41.293565 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-f22e5b4a-7053-44d3-b46e-8a5b05217e46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f53284773f7a430db3d0e16d3c2cd262'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:41.306621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-f22e5b4a-7053-44d3-b46e-8a5b05217e46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f53284773f7a430db3d0e16d3c2cd262'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:41.316171 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-f22e5b4a-7053-44d3-b46e-8a5b05217e46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f53284773f7a430db3d0e16d3c2cd262'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:41.324166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-f22e5b4a-7053-44d3-b46e-8a5b05217e46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f53284773f7a430db3d0e16d3c2cd262'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:41.324872 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-f22e5b4a-7053-44d3-b46e-8a5b05217e46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f53284773f7a430db3d0e16d3c2cd262'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:41.331576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1203/2394] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:42:41 2022] DELETE /identity/v3/users/f53284773f7a430db3d0e16d3c2cd262 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:41.516160 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62962fa1-9913-4444-ab91-c03fd3d2ae10 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:41.530392 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62962fa1-9913-4444-ab91-c03fd3d2ae10 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:41.593339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153794ee-bf4d-49f5-a104-82830781b8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:41.594395 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153794ee-bf4d-49f5-a104-82830781b8bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:41.594685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153794ee-bf4d-49f5-a104-82830781b8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:41.594995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153794ee-bf4d-49f5-a104-82830781b8bc admin admin] PATH_INFO: `/v3/projects/c751cc6b8ac64f3fb7adc9bc71575bd2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:41.596256 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153794ee-bf4d-49f5-a104-82830781b8bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c751cc6b8ac64f3fb7adc9bc71575bd2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:41.598673 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153794ee-bf4d-49f5-a104-82830781b8bc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:41.659798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-153794ee-bf4d-49f5-a104-82830781b8bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c751cc6b8ac64f3fb7adc9bc71575bd2', 'request_id': 'req-153794ee-bf4d-49f5-a104-82830781b8bc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:41.667475 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-153794ee-bf4d-49f5-a104-82830781b8bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c751cc6b8ac64f3fb7adc9bc71575bd2', 'request_id': 'req-153794ee-bf4d-49f5-a104-82830781b8bc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:41.672956 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-153794ee-bf4d-49f5-a104-82830781b8bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c751cc6b8ac64f3fb7adc9bc71575bd2', 'request_id': 'req-153794ee-bf4d-49f5-a104-82830781b8bc'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:41.684352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-153794ee-bf4d-49f5-a104-82830781b8bc admin admin] The token cache is being invalidate because project c751cc6b8ac64f3fb7adc9bc71575bd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:42:41.684586 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-153794ee-bf4d-49f5-a104-82830781b8bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:41.685218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-153794ee-bf4d-49f5-a104-82830781b8bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:41.686738 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1192/2395] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:42:41 2022] DELETE /identity/v3/projects/c751cc6b8ac64f3fb7adc9bc71575bd2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:41.691658 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5459a-f8e9-4166-a480-b8aa074a09d6 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:41.691955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5459a-f8e9-4166-a480-b8aa074a09d6 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:41.692240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5459a-f8e9-4166-a480-b8aa074a09d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:41.710473 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-c9f5459a-f8e9-4166-a480-b8aa074a09d6 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:41.711689 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-c9f5459a-f8e9-4166-a480-b8aa074a09d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:41.770302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1204/2396] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:42:41.782235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038eb3e9-f72a-445c-81f5-14940a2bf885 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:41.782527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038eb3e9-f72a-445c-81f5-14940a2bf885 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:41.782892 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038eb3e9-f72a-445c-81f5-14940a2bf885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:41.803344 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-038eb3e9-f72a-445c-81f5-14940a2bf885 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:41.804566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-038eb3e9-f72a-445c-81f5-14940a2bf885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:41.813867 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1193/2397] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:41.818133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5459a-f8e9-4166-a480-b8aa074a09d6 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:41.829273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5459a-f8e9-4166-a480-b8aa074a09d6 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:41.833819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53005994-7d0a-403c-88ab-177f999e4578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:41.834482 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53005994-7d0a-403c-88ab-177f999e4578 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:41.834799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53005994-7d0a-403c-88ab-177f999e4578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:41.835115 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53005994-7d0a-403c-88ab-177f999e4578 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:41.835844 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53005994-7d0a-403c-88ab-177f999e4578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:42:41.836158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53005994-7d0a-403c-88ab-177f999e4578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:41.837757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53005994-7d0a-403c-88ab-177f999e4578 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:41.853501 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1205/2398] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:42:41.855351 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038eb3e9-f72a-445c-81f5-14940a2bf885 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:41.864858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038eb3e9-f72a-445c-81f5-14940a2bf885 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:41.869622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b659e7-12f3-4e74-a10e-4477f2d11bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:41.871453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b659e7-12f3-4e74-a10e-4477f2d11bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:41.871918 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b659e7-12f3-4e74-a10e-4477f2d11bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:41.872273 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b659e7-12f3-4e74-a10e-4477f2d11bb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:41.873042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b659e7-12f3-4e74-a10e-4477f2d11bb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:41.875362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b659e7-12f3-4e74-a10e-4477f2d11bb0 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:41.913799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1194/2399] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:42:41 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:42:41.918409 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53005994-7d0a-403c-88ab-177f999e4578 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:41.928452 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53005994-7d0a-403c-88ab-177f999e4578 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:41.984829 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c01589-a5e8-4666-b157-10625093dd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:41.985508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c01589-a5e8-4666-b157-10625093dd9e admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:41.985802 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c01589-a5e8-4666-b157-10625093dd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:41.986090 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c01589-a5e8-4666-b157-10625093dd9e admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:41.986804 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c01589-a5e8-4666-b157-10625093dd9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:41.990482 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c01589-a5e8-4666-b157-10625093dd9e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:41.999635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:42.014799 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1206/2400] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:42:41 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:42:42.018903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b659e7-12f3-4e74-a10e-4477f2d11bb0 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:42.029262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b659e7-12f3-4e74-a10e-4477f2d11bb0 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:42.033755 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04782926-401a-462c-8b62-5ca75ccf2521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:42.034413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04782926-401a-462c-8b62-5ca75ccf2521 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:42.034695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04782926-401a-462c-8b62-5ca75ccf2521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:42.035002 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04782926-401a-462c-8b62-5ca75ccf2521 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:42.035728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04782926-401a-462c-8b62-5ca75ccf2521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:42.037303 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04782926-401a-462c-8b62-5ca75ccf2521 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:42.049285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1195/2401] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:42:42.053233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c01589-a5e8-4666-b157-10625093dd9e admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:42.064486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c01589-a5e8-4666-b157-10625093dd9e admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:42.069363 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b79ea1-4f2e-4b17-8082-f268f42ebed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:42.070064 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b79ea1-4f2e-4b17-8082-f268f42ebed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:42.070348 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b79ea1-4f2e-4b17-8082-f268f42ebed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:42.070648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b79ea1-4f2e-4b17-8082-f268f42ebed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:42.071457 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b79ea1-4f2e-4b17-8082-f268f42ebed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:42.073093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b79ea1-4f2e-4b17-8082-f268f42ebed7 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:42.084705 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1207/2402] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:42:42.088958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04782926-401a-462c-8b62-5ca75ccf2521 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:42.098878 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04782926-401a-462c-8b62-5ca75ccf2521 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:42.103776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60eb52-4c9f-47d2-ac0f-f9d052c5de39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:42.105237 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60eb52-4c9f-47d2-ac0f-f9d052c5de39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:42.105610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60eb52-4c9f-47d2-ac0f-f9d052c5de39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:42.105914 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60eb52-4c9f-47d2-ac0f-f9d052c5de39 admin admin] PATH_INFO: `/v3/projects/ac82860a551b4ef8ad85d5a4600bca30/users/5541c49aa5414afaba6997f8386a5612/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:42.122899 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60eb52-4c9f-47d2-ac0f-f9d052c5de39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac82860a551b4ef8ad85d5a4600bca30, user_id=5541c49aa5414afaba6997f8386a5612, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:42.125618 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60eb52-4c9f-47d2-ac0f-f9d052c5de39 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:42.141819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1196/2403] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:42:42 2022] PUT /identity/v3/projects/ac82860a551b4ef8ad85d5a4600bca30/users/5541c49aa5414afaba6997f8386a5612/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:42.153766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448ce61-0b8d-4fb7-9474-58978b14775d None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:42.154233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448ce61-0b8d-4fb7-9474-58978b14775d None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:42.154633 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448ce61-0b8d-4fb7-9474-58978b14775d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:42.183797 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-f448ce61-0b8d-4fb7-9474-58978b14775d None None] MFA Rules not processed for user `5541c49aa5414afaba6997f8386a5612`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:42.185007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-f448ce61-0b8d-4fb7-9474-58978b14775d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:42.256611 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1208/2404] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:42:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:42.268683 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60eb52-4c9f-47d2-ac0f-f9d052c5de39 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:42.271616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1c60eb52-4c9f-47d2-ac0f-f9d052c5de39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:42.279322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:42.351398 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60eb52-4c9f-47d2-ac0f-f9d052c5de39 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:42.357031 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b249b1-4bf3-4961-8d84-13e15190d9b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:42.357692 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b249b1-4bf3-4961-8d84-13e15190d9b6 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:42.357987 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b249b1-4bf3-4961-8d84-13e15190d9b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:42.358274 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b249b1-4bf3-4961-8d84-13e15190d9b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:42.366070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b249b1-4bf3-4961-8d84-13e15190d9b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:42.367684 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b249b1-4bf3-4961-8d84-13e15190d9b6 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:42.376612 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1197/2405] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:42:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:42:42.970154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03358e-b0a8-43ed-b85e-a08234cf2198 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:42.970452 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03358e-b0a8-43ed-b85e-a08234cf2198 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:42.970736 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03358e-b0a8-43ed-b85e-a08234cf2198 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:42.972298 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1209/2406] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 22:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:42:42.977986 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727c60b-a026-422e-a13f-2119f134513d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:42.978499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727c60b-a026-422e-a13f-2119f134513d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:42.978702 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727c60b-a026-422e-a13f-2119f134513d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:42.998301 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c727c60b-a026-422e-a13f-2119f134513d None None] MFA Rules not processed for user `37ba527b96634fc2a48b6ca9597d8191`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:42.999590 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c727c60b-a026-422e-a13f-2119f134513d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:43.053780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1198/2407] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 22:42:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:43.067161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03358e-b0a8-43ed-b85e-a08234cf2198 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:43.068905 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-ce03358e-b0a8-43ed-b85e-a08234cf2198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:43.082548 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03358e-b0a8-43ed-b85e-a08234cf2198 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:43.086126 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f1e17-edc0-44e2-bb2d-964069e1db41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:43.086781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f1e17-edc0-44e2-bb2d-964069e1db41 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:43.087106 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f1e17-edc0-44e2-bb2d-964069e1db41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:43.087395 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f1e17-edc0-44e2-bb2d-964069e1db41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:43.095478 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f1e17-edc0-44e2-bb2d-964069e1db41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:43.096918 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f1e17-edc0-44e2-bb2d-964069e1db41 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:43.105188 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1210/2408] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:42:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:42:43.188544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3a876-700e-4f26-aa0d-343482000843 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:43.188843 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3a876-700e-4f26-aa0d-343482000843 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:43.189132 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3a876-700e-4f26-aa0d-343482000843 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:43.190545 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1199/2409] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 22:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 22:42:43.195758 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a3551-aa43-46e4-a276-33ac078cc829 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:43.196053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a3551-aa43-46e4-a276-33ac078cc829 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:43.196333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a3551-aa43-46e4-a276-33ac078cc829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:43.233497 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-8c7a3551-aa43-46e4-a276-33ac078cc829 None None] MFA Rules not processed for user `35d686923ef541819de3c77847d38511`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:43.234697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8c7a3551-aa43-46e4-a276-33ac078cc829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:43.244207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1211/2410] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 22:42:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:43.255406 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3a876-700e-4f26-aa0d-343482000843 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:43.267566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3a876-700e-4f26-aa0d-343482000843 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:43.271456 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4b028-2c2f-422e-9191-1181013afa61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:43.272097 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4b028-2c2f-422e-9191-1181013afa61 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:43.272385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4b028-2c2f-422e-9191-1181013afa61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:43.272668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4b028-2c2f-422e-9191-1181013afa61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:43.283540 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4b028-2c2f-422e-9191-1181013afa61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:43.284901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c4b028-2c2f-422e-9191-1181013afa61 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:43.296100 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1200/2411] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:42:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:42:50.718587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a3551-aa43-46e4-a276-33ac078cc829 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:50.791021 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a3551-aa43-46e4-a276-33ac078cc829 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:50.796910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be445a26-ef77-4b51-a94a-e55e1a8b3111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:50.798087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be445a26-ef77-4b51-a94a-e55e1a8b3111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:50.798527 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be445a26-ef77-4b51-a94a-e55e1a8b3111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:50.798859 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be445a26-ef77-4b51-a94a-e55e1a8b3111 admin admin] PATH_INFO: `/v3/users/5541c49aa5414afaba6997f8386a5612` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:50.800484 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be445a26-ef77-4b51-a94a-e55e1a8b3111 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5541c49aa5414afaba6997f8386a5612)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:50.802623 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be445a26-ef77-4b51-a94a-e55e1a8b3111 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:50.850060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:50.860993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-be445a26-ef77-4b51-a94a-e55e1a8b3111 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5541c49aa5414afaba6997f8386a5612'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:50.874063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-be445a26-ef77-4b51-a94a-e55e1a8b3111 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5541c49aa5414afaba6997f8386a5612'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:50.887508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-be445a26-ef77-4b51-a94a-e55e1a8b3111 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5541c49aa5414afaba6997f8386a5612'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:50.894407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-be445a26-ef77-4b51-a94a-e55e1a8b3111 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5541c49aa5414afaba6997f8386a5612'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:50.895071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-be445a26-ef77-4b51-a94a-e55e1a8b3111 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5541c49aa5414afaba6997f8386a5612'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:50.902055 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1212/2412] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:42:50 2022] DELETE /identity/v3/users/5541c49aa5414afaba6997f8386a5612 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:50.907101 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0691611-ef77-4f75-90e9-f60e3fc71dc1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:50.907619 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0691611-ef77-4f75-90e9-f60e3fc71dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:50.907764 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0691611-ef77-4f75-90e9-f60e3fc71dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:50.925630 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c0691611-ef77-4f75-90e9-f60e3fc71dc1 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:50.926822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c0691611-ef77-4f75-90e9-f60e3fc71dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:50.991332 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1201/2413] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:42:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:51.001764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be445a26-ef77-4b51-a94a-e55e1a8b3111 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:51.003596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-be445a26-ef77-4b51-a94a-e55e1a8b3111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:51.062189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be445a26-ef77-4b51-a94a-e55e1a8b3111 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:51.065837 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd7872-521e-4e2f-99f0-bee7ae4e4196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:51.066489 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd7872-521e-4e2f-99f0-bee7ae4e4196 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:51.066777 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd7872-521e-4e2f-99f0-bee7ae4e4196 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:51.067094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd7872-521e-4e2f-99f0-bee7ae4e4196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:51.076731 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd7872-521e-4e2f-99f0-bee7ae4e4196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:51.078198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd7872-521e-4e2f-99f0-bee7ae4e4196 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:51.087679 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1213/2414] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:42:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:42:51.413240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0691611-ef77-4f75-90e9-f60e3fc71dc1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:51.414714 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c0691611-ef77-4f75-90e9-f60e3fc71dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:51.429533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0691611-ef77-4f75-90e9-f60e3fc71dc1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:51.435235 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb10bea-ceb2-4143-b022-3b9962184011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:51.436095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb10bea-ceb2-4143-b022-3b9962184011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:51.436473 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb10bea-ceb2-4143-b022-3b9962184011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:51.436668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb10bea-ceb2-4143-b022-3b9962184011 admin admin] PATH_INFO: `/v3/projects/ac82860a551b4ef8ad85d5a4600bca30` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:51.437880 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb10bea-ceb2-4143-b022-3b9962184011 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac82860a551b4ef8ad85d5a4600bca30)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:51.441206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb10bea-ceb2-4143-b022-3b9962184011 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:51.469581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:51.497304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4eb10bea-ceb2-4143-b022-3b9962184011 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac82860a551b4ef8ad85d5a4600bca30', 'request_id': 'req-4eb10bea-ceb2-4143-b022-3b9962184011'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:51.505188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4eb10bea-ceb2-4143-b022-3b9962184011 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac82860a551b4ef8ad85d5a4600bca30', 'request_id': 'req-4eb10bea-ceb2-4143-b022-3b9962184011'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:51.512183 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4eb10bea-ceb2-4143-b022-3b9962184011 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac82860a551b4ef8ad85d5a4600bca30', 'request_id': 'req-4eb10bea-ceb2-4143-b022-3b9962184011'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:51.525000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4eb10bea-ceb2-4143-b022-3b9962184011 admin admin] The token cache is being invalidate because project ac82860a551b4ef8ad85d5a4600bca30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:42:51.525299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4eb10bea-ceb2-4143-b022-3b9962184011 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:51.526604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4eb10bea-ceb2-4143-b022-3b9962184011 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:51.533427 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1202/2415] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:42:51 2022] DELETE /identity/v3/projects/ac82860a551b4ef8ad85d5a4600bca30 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:51.535071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505d706-4a6f-4570-b161-50e143246588 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:51.535365 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505d706-4a6f-4570-b161-50e143246588 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:51.535651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505d706-4a6f-4570-b161-50e143246588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:51.556224 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-b505d706-4a6f-4570-b161-50e143246588 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:51.557559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b505d706-4a6f-4570-b161-50e143246588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:51.622395 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1214/2416] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:51.630389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b06c01-8886-4d6f-ae90-d14acb9b2c5e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:51.630684 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b06c01-8886-4d6f-ae90-d14acb9b2c5e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:51.631008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b06c01-8886-4d6f-ae90-d14acb9b2c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:51.649155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e5b06c01-8886-4d6f-ae90-d14acb9b2c5e None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:51.650365 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e5b06c01-8886-4d6f-ae90-d14acb9b2c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:51.659689 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1203/2417] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:51.663825 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505d706-4a6f-4570-b161-50e143246588 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:51.675469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505d706-4a6f-4570-b161-50e143246588 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:51.681511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1eca4-d02c-44f2-9731-f7098166a1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:51.682679 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1eca4-d02c-44f2-9731-f7098166a1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:51.682795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1eca4-d02c-44f2-9731-f7098166a1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:51.683088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1eca4-d02c-44f2-9731-f7098166a1d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:51.683823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1eca4-d02c-44f2-9731-f7098166a1d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:42:51.684123 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1eca4-d02c-44f2-9731-f7098166a1d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:51.685714 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1eca4-d02c-44f2-9731-f7098166a1d8 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:51.701862 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1215/2418] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:42:51.703387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b06c01-8886-4d6f-ae90-d14acb9b2c5e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:51.712888 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b06c01-8886-4d6f-ae90-d14acb9b2c5e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:51.718080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd27ca-ba1e-44f6-afa7-e322f456f2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:51.718745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd27ca-ba1e-44f6-afa7-e322f456f2d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:51.719072 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd27ca-ba1e-44f6-afa7-e322f456f2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:51.719362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbd27ca-ba1e-44f6-afa7-e322f456f2d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:51.720064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd27ca-ba1e-44f6-afa7-e322f456f2d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:51.721968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbd27ca-ba1e-44f6-afa7-e322f456f2d7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:51.749511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1204/2419] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:42:51 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:42:51.753241 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1eca4-d02c-44f2-9731-f7098166a1d8 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:51.764105 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1eca4-d02c-44f2-9731-f7098166a1d8 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:51.806498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:51.824395 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73184a6-83d3-49a7-8895-ca26bbd52fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:51.825309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73184a6-83d3-49a7-8895-ca26bbd52fa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:51.825685 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73184a6-83d3-49a7-8895-ca26bbd52fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:51.825969 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73184a6-83d3-49a7-8895-ca26bbd52fa6 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:51.826651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73184a6-83d3-49a7-8895-ca26bbd52fa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:51.828609 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73184a6-83d3-49a7-8895-ca26bbd52fa6 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:51.855999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1216/2420] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:42:51 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:42:51.859983 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd27ca-ba1e-44f6-afa7-e322f456f2d7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:51.872553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbd27ca-ba1e-44f6-afa7-e322f456f2d7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:51.877639 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649d1a4-3164-4352-a4cc-fb3d02b6f8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:51.878457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649d1a4-3164-4352-a4cc-fb3d02b6f8ab admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:51.878716 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649d1a4-3164-4352-a4cc-fb3d02b6f8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:51.879058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649d1a4-3164-4352-a4cc-fb3d02b6f8ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:51.879894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649d1a4-3164-4352-a4cc-fb3d02b6f8ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:51.881498 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649d1a4-3164-4352-a4cc-fb3d02b6f8ab admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:51.892525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1205/2421] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:42:51.896626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73184a6-83d3-49a7-8895-ca26bbd52fa6 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:51.906857 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73184a6-83d3-49a7-8895-ca26bbd52fa6 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:51.911661 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7158e35-7b60-4094-81b2-578d42dd0f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:51.912381 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7158e35-7b60-4094-81b2-578d42dd0f88 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:51.912628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7158e35-7b60-4094-81b2-578d42dd0f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:51.912910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7158e35-7b60-4094-81b2-578d42dd0f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:51.913638 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7158e35-7b60-4094-81b2-578d42dd0f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:51.915295 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7158e35-7b60-4094-81b2-578d42dd0f88 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:51.926360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1217/2422] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:42:51.930853 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649d1a4-3164-4352-a4cc-fb3d02b6f8ab admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:51.947258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649d1a4-3164-4352-a4cc-fb3d02b6f8ab admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:51.952602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1bbed-387b-4c03-adf8-73770572c134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:51.953522 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1bbed-387b-4c03-adf8-73770572c134 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:51.953806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1bbed-387b-4c03-adf8-73770572c134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:51.954081 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1bbed-387b-4c03-adf8-73770572c134 admin admin] PATH_INFO: `/v3/projects/3ff91b9df01440cd88b0d9a39a307d31/users/14937cbc49fc4b61beb7f3500c8c5f15/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:51.974744 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1bbed-387b-4c03-adf8-73770572c134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ff91b9df01440cd88b0d9a39a307d31, user_id=14937cbc49fc4b61beb7f3500c8c5f15, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:51.977519 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1bbed-387b-4c03-adf8-73770572c134 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:51.998113 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1206/2423] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:42:51 2022] PUT /identity/v3/projects/3ff91b9df01440cd88b0d9a39a307d31/users/14937cbc49fc4b61beb7f3500c8c5f15/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:42:52.008963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9dc4f-cefd-4fb1-84ca-56abbb4a4fbd None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:52.009263 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9dc4f-cefd-4fb1-84ca-56abbb4a4fbd None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:52.009566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9dc4f-cefd-4fb1-84ca-56abbb4a4fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:52.042586 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-7fe9dc4f-cefd-4fb1-84ca-56abbb4a4fbd None None] MFA Rules not processed for user `14937cbc49fc4b61beb7f3500c8c5f15`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:52.043945 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7fe9dc4f-cefd-4fb1-84ca-56abbb4a4fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:52.134563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1218/2424] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:42:52 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:52.139916 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1bbed-387b-4c03-adf8-73770572c134 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:52.205900 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1bbed-387b-4c03-adf8-73770572c134 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:52.210513 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbf125-3e81-4d61-8ece-4ac68897ad79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:52.211263 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbf125-3e81-4d61-8ece-4ac68897ad79 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:52.211547 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbf125-3e81-4d61-8ece-4ac68897ad79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:52.211829 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbf125-3e81-4d61-8ece-4ac68897ad79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:52.212520 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbf125-3e81-4d61-8ece-4ac68897ad79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:52.214393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbf125-3e81-4d61-8ece-4ac68897ad79 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:52.246622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1207/2425] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:42:52 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:42:52.249362 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9dc4f-cefd-4fb1-84ca-56abbb4a4fbd None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:52.261340 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9dc4f-cefd-4fb1-84ca-56abbb4a4fbd None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:52.320213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ea1ae-fc80-4c68-bc78-e246dadd753c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:52.320904 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ea1ae-fc80-4c68-bc78-e246dadd753c admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:52.321225 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ea1ae-fc80-4c68-bc78-e246dadd753c admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:52.323225 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ea1ae-fc80-4c68-bc78-e246dadd753c admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:52.323225 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ea1ae-fc80-4c68-bc78-e246dadd753c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:52.324986 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ea1ae-fc80-4c68-bc78-e246dadd753c admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:52.352177 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1219/2426] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:42:52 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:42:52.355311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbf125-3e81-4d61-8ece-4ac68897ad79 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:52.366404 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbf125-3e81-4d61-8ece-4ac68897ad79 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:52.371926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8e4d6-f194-4fa2-9329-9cc1bd686dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:52.372980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8e4d6-f194-4fa2-9329-9cc1bd686dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:52.373377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8e4d6-f194-4fa2-9329-9cc1bd686dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:52.373721 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8e4d6-f194-4fa2-9329-9cc1bd686dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:52.374526 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8e4d6-f194-4fa2-9329-9cc1bd686dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:52.376241 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8e4d6-f194-4fa2-9329-9cc1bd686dc9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:52.387996 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1208/2427] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:42:52.392774 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ea1ae-fc80-4c68-bc78-e246dadd753c admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:52.402727 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ea1ae-fc80-4c68-bc78-e246dadd753c admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:52.407547 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81e749-ca85-4716-875d-fe45196cea7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:52.408440 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81e749-ca85-4716-875d-fe45196cea7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:52.408730 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81e749-ca85-4716-875d-fe45196cea7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:52.409017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81e749-ca85-4716-875d-fe45196cea7e admin admin] PATH_INFO: `/v3/projects/ae106b24052747a39af71e0c02ff94a5/users/41e737ece4d345ee889f1b4256f2842a/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:52.425488 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81e749-ca85-4716-875d-fe45196cea7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae106b24052747a39af71e0c02ff94a5, user_id=41e737ece4d345ee889f1b4256f2842a, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:52.428192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81e749-ca85-4716-875d-fe45196cea7e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:52.443238 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1220/2428] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:42:52 2022] PUT /identity/v3/projects/ae106b24052747a39af71e0c02ff94a5/users/41e737ece4d345ee889f1b4256f2842a/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:52.447167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8e4d6-f194-4fa2-9329-9cc1bd686dc9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:52.457977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8e4d6-f194-4fa2-9329-9cc1bd686dc9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:52.477421 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:52.521231 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58eff5-64b1-45b1-9a2b-4358d0f68987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:52.521903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58eff5-64b1-45b1-9a2b-4358d0f68987 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:52.522193 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58eff5-64b1-45b1-9a2b-4358d0f68987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:52.522478 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58eff5-64b1-45b1-9a2b-4358d0f68987 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:52.523233 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58eff5-64b1-45b1-9a2b-4358d0f68987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:52.524824 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58eff5-64b1-45b1-9a2b-4358d0f68987 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:52.535277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1209/2429] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:42:52.539462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81e749-ca85-4716-875d-fe45196cea7e admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:52.550194 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81e749-ca85-4716-875d-fe45196cea7e admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:52.554874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7847f7d-9a1e-4442-993a-fa7cfdaf24c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:52.555785 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7847f7d-9a1e-4442-993a-fa7cfdaf24c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:52.556074 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7847f7d-9a1e-4442-993a-fa7cfdaf24c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:52.556357 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7847f7d-9a1e-4442-993a-fa7cfdaf24c8 admin admin] PATH_INFO: `/v3/projects/ae106b24052747a39af71e0c02ff94a5/users/41e737ece4d345ee889f1b4256f2842a/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:52.558307 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7847f7d-9a1e-4442-993a-fa7cfdaf24c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae106b24052747a39af71e0c02ff94a5, user_id=41e737ece4d345ee889f1b4256f2842a, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:52.561390 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7847f7d-9a1e-4442-993a-fa7cfdaf24c8 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:52.572157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1221/2430] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:42:52 2022] PUT /identity/v3/projects/ae106b24052747a39af71e0c02ff94a5/users/41e737ece4d345ee889f1b4256f2842a/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:52.585709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78837fe6-ef46-46de-aaae-9165c8543c2a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:52.586191 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78837fe6-ef46-46de-aaae-9165c8543c2a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:52.586656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78837fe6-ef46-46de-aaae-9165c8543c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:52.617653 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-78837fe6-ef46-46de-aaae-9165c8543c2a None None] MFA Rules not processed for user `41e737ece4d345ee889f1b4256f2842a`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:52.618947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-78837fe6-ef46-46de-aaae-9165c8543c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:52.693115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1210/2431] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:42:52 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:42:52.704424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7847f7d-9a1e-4442-993a-fa7cfdaf24c8 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:52.706098 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b7847f7d-9a1e-4442-993a-fa7cfdaf24c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:52.712859 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:52.770025 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7847f7d-9a1e-4442-993a-fa7cfdaf24c8 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:52.773936 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72733c5-d93e-45f2-a484-f1e0d5e0d514 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:52.774586 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72733c5-d93e-45f2-a484-f1e0d5e0d514 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:52.774874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72733c5-d93e-45f2-a484-f1e0d5e0d514 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:52.775190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72733c5-d93e-45f2-a484-f1e0d5e0d514 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:52.785532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72733c5-d93e-45f2-a484-f1e0d5e0d514 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:52.786998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72733c5-d93e-45f2-a484-f1e0d5e0d514 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:52.796668 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1222/2432] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:42:52 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:42:54.201828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78837fe6-ef46-46de-aaae-9165c8543c2a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:54.268033 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78837fe6-ef46-46de-aaae-9165c8543c2a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:54.272528 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd70a8f-0ccb-4b9e-b16d-592fb5b60038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:54.273394 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd70a8f-0ccb-4b9e-b16d-592fb5b60038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:54.273677 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd70a8f-0ccb-4b9e-b16d-592fb5b60038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:54.273953 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd70a8f-0ccb-4b9e-b16d-592fb5b60038 admin admin] PATH_INFO: `/v3/users/14937cbc49fc4b61beb7f3500c8c5f15` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:54.275296 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd70a8f-0ccb-4b9e-b16d-592fb5b60038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14937cbc49fc4b61beb7f3500c8c5f15)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:54.277434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd70a8f-0ccb-4b9e-b16d-592fb5b60038 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:54.322088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:54.327059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4fd70a8f-0ccb-4b9e-b16d-592fb5b60038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14937cbc49fc4b61beb7f3500c8c5f15'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:54.334712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4fd70a8f-0ccb-4b9e-b16d-592fb5b60038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14937cbc49fc4b61beb7f3500c8c5f15'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:54.343420 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4fd70a8f-0ccb-4b9e-b16d-592fb5b60038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14937cbc49fc4b61beb7f3500c8c5f15'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:54.344058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4fd70a8f-0ccb-4b9e-b16d-592fb5b60038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14937cbc49fc4b61beb7f3500c8c5f15'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:54.344675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4fd70a8f-0ccb-4b9e-b16d-592fb5b60038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14937cbc49fc4b61beb7f3500c8c5f15'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:54.362606 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1211/2433] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:42:54 2022] DELETE /identity/v3/users/14937cbc49fc4b61beb7f3500c8c5f15 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:54.367190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a871fb9-a609-4c46-bb3a-887dbf650874 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:54.367648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a871fb9-a609-4c46-bb3a-887dbf650874 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:54.367760 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a871fb9-a609-4c46-bb3a-887dbf650874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:54.385294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-2a871fb9-a609-4c46-bb3a-887dbf650874 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:54.386510 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2a871fb9-a609-4c46-bb3a-887dbf650874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:54.446476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1223/2434] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:54.456871 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd70a8f-0ccb-4b9e-b16d-592fb5b60038 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:54.458266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4fd70a8f-0ccb-4b9e-b16d-592fb5b60038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:54.520732 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd70a8f-0ccb-4b9e-b16d-592fb5b60038 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:54.527168 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db3d15-f47b-4b6e-b4e0-c63dbabd45a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:54.528201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db3d15-f47b-4b6e-b4e0-c63dbabd45a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:54.528369 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db3d15-f47b-4b6e-b4e0-c63dbabd45a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:54.528651 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db3d15-f47b-4b6e-b4e0-c63dbabd45a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:54.553080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db3d15-f47b-4b6e-b4e0-c63dbabd45a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:54.554567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db3d15-f47b-4b6e-b4e0-c63dbabd45a6 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:54.565592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1212/2435] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:42:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:42:54.953283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a871fb9-a609-4c46-bb3a-887dbf650874 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:54.956130 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2a871fb9-a609-4c46-bb3a-887dbf650874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:54.977684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a871fb9-a609-4c46-bb3a-887dbf650874 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:54.983929 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5a483-9abe-461a-8b09-d8d8d7e06026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:54.984800 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5a483-9abe-461a-8b09-d8d8d7e06026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:54.985088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5a483-9abe-461a-8b09-d8d8d7e06026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:54.985368 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5a483-9abe-461a-8b09-d8d8d7e06026 admin admin] PATH_INFO: `/v3/projects/3ff91b9df01440cd88b0d9a39a307d31` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:54.986653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5a483-9abe-461a-8b09-d8d8d7e06026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ff91b9df01440cd88b0d9a39a307d31)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:54.988780 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5a483-9abe-461a-8b09-d8d8d7e06026 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:55.042639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-3ac5a483-9abe-461a-8b09-d8d8d7e06026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ff91b9df01440cd88b0d9a39a307d31', 'request_id': 'req-3ac5a483-9abe-461a-8b09-d8d8d7e06026'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:55.049865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-3ac5a483-9abe-461a-8b09-d8d8d7e06026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ff91b9df01440cd88b0d9a39a307d31', 'request_id': 'req-3ac5a483-9abe-461a-8b09-d8d8d7e06026'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:55.059727 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-3ac5a483-9abe-461a-8b09-d8d8d7e06026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ff91b9df01440cd88b0d9a39a307d31', 'request_id': 'req-3ac5a483-9abe-461a-8b09-d8d8d7e06026'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:55.073061 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-3ac5a483-9abe-461a-8b09-d8d8d7e06026 admin admin] The token cache is being invalidate because project 3ff91b9df01440cd88b0d9a39a307d31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:42:55.073476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-3ac5a483-9abe-461a-8b09-d8d8d7e06026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:55.074272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-3ac5a483-9abe-461a-8b09-d8d8d7e06026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:55.075906 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1224/2436] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:42:54 2022] DELETE /identity/v3/projects/3ff91b9df01440cd88b0d9a39a307d31 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:55.080592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db3d15-f47b-4b6e-b4e0-c63dbabd45a6 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:55.090513 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db3d15-f47b-4b6e-b4e0-c63dbabd45a6 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:55.153372 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01a4d1-e84a-479f-b06d-f70f6718cf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:55.154184 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01a4d1-e84a-479f-b06d-f70f6718cf4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:55.154477 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01a4d1-e84a-479f-b06d-f70f6718cf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:55.154763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01a4d1-e84a-479f-b06d-f70f6718cf4b admin admin] PATH_INFO: `/v3/users/41e737ece4d345ee889f1b4256f2842a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:55.156019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01a4d1-e84a-479f-b06d-f70f6718cf4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=41e737ece4d345ee889f1b4256f2842a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:55.157939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01a4d1-e84a-479f-b06d-f70f6718cf4b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:55.207195 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2e01a4d1-e84a-479f-b06d-f70f6718cf4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41e737ece4d345ee889f1b4256f2842a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:55.215649 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2e01a4d1-e84a-479f-b06d-f70f6718cf4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41e737ece4d345ee889f1b4256f2842a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:55.224018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2e01a4d1-e84a-479f-b06d-f70f6718cf4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41e737ece4d345ee889f1b4256f2842a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:55.224649 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2e01a4d1-e84a-479f-b06d-f70f6718cf4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41e737ece4d345ee889f1b4256f2842a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:55.225258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2e01a4d1-e84a-479f-b06d-f70f6718cf4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41e737ece4d345ee889f1b4256f2842a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:55.227170 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:55.246799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1213/2437] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:42:55 2022] DELETE /identity/v3/users/41e737ece4d345ee889f1b4256f2842a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:55.540019 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5a483-9abe-461a-8b09-d8d8d7e06026 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:55.541691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3ac5a483-9abe-461a-8b09-d8d8d7e06026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:55.560217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5a483-9abe-461a-8b09-d8d8d7e06026 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:55.598451 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:55.628642 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a3d8a-a9bf-4250-ae70-83e8d6291656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:55.629515 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a3d8a-a9bf-4250-ae70-83e8d6291656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:55.629823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a3d8a-a9bf-4250-ae70-83e8d6291656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:55.630089 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a3d8a-a9bf-4250-ae70-83e8d6291656 admin admin] PATH_INFO: `/v3/projects/ae106b24052747a39af71e0c02ff94a5` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:55.631329 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a3d8a-a9bf-4250-ae70-83e8d6291656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae106b24052747a39af71e0c02ff94a5)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:55.633251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a3d8a-a9bf-4250-ae70-83e8d6291656 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:55.683610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-202a3d8a-a9bf-4250-ae70-83e8d6291656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae106b24052747a39af71e0c02ff94a5', 'request_id': 'req-202a3d8a-a9bf-4250-ae70-83e8d6291656'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:55.689798 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-202a3d8a-a9bf-4250-ae70-83e8d6291656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae106b24052747a39af71e0c02ff94a5', 'request_id': 'req-202a3d8a-a9bf-4250-ae70-83e8d6291656'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:55.701732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-202a3d8a-a9bf-4250-ae70-83e8d6291656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae106b24052747a39af71e0c02ff94a5', 'request_id': 'req-202a3d8a-a9bf-4250-ae70-83e8d6291656'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:55.712764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-202a3d8a-a9bf-4250-ae70-83e8d6291656 admin admin] The token cache is being invalidate because project ae106b24052747a39af71e0c02ff94a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:42:55.713229 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-202a3d8a-a9bf-4250-ae70-83e8d6291656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:55.713939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-202a3d8a-a9bf-4250-ae70-83e8d6291656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:55.716794 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1225/2438] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:42:55 2022] DELETE /identity/v3/projects/ae106b24052747a39af71e0c02ff94a5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:55.722073 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504bfa7-752a-424c-b745-53142b442688 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:55.722370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504bfa7-752a-424c-b745-53142b442688 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:55.722657 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504bfa7-752a-424c-b745-53142b442688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:55.741361 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-6504bfa7-752a-424c-b745-53142b442688 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:55.742492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-6504bfa7-752a-424c-b745-53142b442688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:55.807244 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1214/2439] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:55.816372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afa91b-426f-45e9-9852-5a678dd6a177 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:55.816665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afa91b-426f-45e9-9852-5a678dd6a177 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:55.817132 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afa91b-426f-45e9-9852-5a678dd6a177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:55.840875 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-18afa91b-426f-45e9-9852-5a678dd6a177 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:55.843093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-18afa91b-426f-45e9-9852-5a678dd6a177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:55.851732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1226/2440] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:55.857242 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504bfa7-752a-424c-b745-53142b442688 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:55.867017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504bfa7-752a-424c-b745-53142b442688 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:55.872269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4bfe8-fe0f-4924-979a-b25957ef7b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:55.872930 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4bfe8-fe0f-4924-979a-b25957ef7b80 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:55.873219 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4bfe8-fe0f-4924-979a-b25957ef7b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:55.873503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4bfe8-fe0f-4924-979a-b25957ef7b80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:55.874232 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4bfe8-fe0f-4924-979a-b25957ef7b80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:42:55.874546 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4bfe8-fe0f-4924-979a-b25957ef7b80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:55.876724 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4bfe8-fe0f-4924-979a-b25957ef7b80 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:55.888811 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1215/2441] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:42:55.893024 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18afa91b-426f-45e9-9852-5a678dd6a177 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:55.902966 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18afa91b-426f-45e9-9852-5a678dd6a177 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:55.908533 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857fe49-2413-4731-86ff-fe2e296ea853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:55.909209 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857fe49-2413-4731-86ff-fe2e296ea853 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:55.909497 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857fe49-2413-4731-86ff-fe2e296ea853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:55.909779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857fe49-2413-4731-86ff-fe2e296ea853 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:55.910486 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857fe49-2413-4731-86ff-fe2e296ea853 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:55.912439 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b857fe49-2413-4731-86ff-fe2e296ea853 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:55.954686 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1227/2442] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:42:55 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:42:55.958727 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4bfe8-fe0f-4924-979a-b25957ef7b80 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:55.969991 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4bfe8-fe0f-4924-979a-b25957ef7b80 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:56.039256 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a3bc9-1084-427b-94e3-9b087be984d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:56.039942 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a3bc9-1084-427b-94e3-9b087be984d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:56.040231 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a3bc9-1084-427b-94e3-9b087be984d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:56.040774 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a3bc9-1084-427b-94e3-9b087be984d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:56.041482 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a3bc9-1084-427b-94e3-9b087be984d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:56.043949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a3bc9-1084-427b-94e3-9b087be984d9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:56.071095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1216/2443] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:42:55 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:42:56.076125 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857fe49-2413-4731-86ff-fe2e296ea853 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:56.098474 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857fe49-2413-4731-86ff-fe2e296ea853 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:56.104697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1ffde-7078-4384-89cd-0cbb92e61128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:56.105821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1ffde-7078-4384-89cd-0cbb92e61128 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:56.105821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1ffde-7078-4384-89cd-0cbb92e61128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:56.106114 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1ffde-7078-4384-89cd-0cbb92e61128 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:56.106852 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a1ffde-7078-4384-89cd-0cbb92e61128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:56.108496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a1ffde-7078-4384-89cd-0cbb92e61128 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:56.121777 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1228/2444] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:42:56.124611 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a3bc9-1084-427b-94e3-9b087be984d9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:56.139200 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a3bc9-1084-427b-94e3-9b087be984d9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:56.144068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d03775-ec03-4d0a-9bbd-51f30e5c43fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:56.144749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d03775-ec03-4d0a-9bbd-51f30e5c43fd admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:56.145040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d03775-ec03-4d0a-9bbd-51f30e5c43fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:56.145324 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d03775-ec03-4d0a-9bbd-51f30e5c43fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:56.146051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d03775-ec03-4d0a-9bbd-51f30e5c43fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:56.147698 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d03775-ec03-4d0a-9bbd-51f30e5c43fd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:56.159573 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1217/2445] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:42:56.163660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1ffde-7078-4384-89cd-0cbb92e61128 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:56.174467 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1ffde-7078-4384-89cd-0cbb92e61128 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:56.179341 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049164f-cdc8-4f60-a5bc-b5ee6f3ae75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:56.180342 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049164f-cdc8-4f60-a5bc-b5ee6f3ae75a admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:56.180693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049164f-cdc8-4f60-a5bc-b5ee6f3ae75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:56.181079 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5049164f-cdc8-4f60-a5bc-b5ee6f3ae75a admin admin] PATH_INFO: `/v3/projects/3a700048d1f44133bf1d76f5494db985/users/ab22232eb33f4b278ac58058278db05d/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:56.198387 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5049164f-cdc8-4f60-a5bc-b5ee6f3ae75a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a700048d1f44133bf1d76f5494db985, user_id=ab22232eb33f4b278ac58058278db05d, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:56.201778 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5049164f-cdc8-4f60-a5bc-b5ee6f3ae75a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:56.214460 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1229/2446] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:42:56 2022] PUT /identity/v3/projects/3a700048d1f44133bf1d76f5494db985/users/ab22232eb33f4b278ac58058278db05d/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:56.225499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e35b96-24c0-4864-86cc-d1f254583993 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:56.225989 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e35b96-24c0-4864-86cc-d1f254583993 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:56.226087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e35b96-24c0-4864-86cc-d1f254583993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:56.253414 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-e8e35b96-24c0-4864-86cc-d1f254583993 None None] MFA Rules not processed for user `ab22232eb33f4b278ac58058278db05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:56.254638 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e8e35b96-24c0-4864-86cc-d1f254583993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:56.296188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:56.324948 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1218/2447] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:42:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:56.329957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049164f-cdc8-4f60-a5bc-b5ee6f3ae75a admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:56.388496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:56.392578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5049164f-cdc8-4f60-a5bc-b5ee6f3ae75a admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:56.404763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010e4d3-0544-457c-aa8c-90eef2d9b05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:56.406683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010e4d3-0544-457c-aa8c-90eef2d9b05b admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:56.406683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010e4d3-0544-457c-aa8c-90eef2d9b05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:56.406683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010e4d3-0544-457c-aa8c-90eef2d9b05b admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:56.407174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010e4d3-0544-457c-aa8c-90eef2d9b05b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:56.409319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010e4d3-0544-457c-aa8c-90eef2d9b05b admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:56.442814 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1230/2448] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:42:56 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:42:56.443883 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e35b96-24c0-4864-86cc-d1f254583993 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:56.457465 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e35b96-24c0-4864-86cc-d1f254583993 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:56.517011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22a802-3ad6-4a10-83d2-27aca44a5c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:56.517692 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22a802-3ad6-4a10-83d2-27aca44a5c2c admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:56.518093 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22a802-3ad6-4a10-83d2-27aca44a5c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:56.518262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22a802-3ad6-4a10-83d2-27aca44a5c2c admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:56.519020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e22a802-3ad6-4a10-83d2-27aca44a5c2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:56.520907 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e22a802-3ad6-4a10-83d2-27aca44a5c2c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:56.544802 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1219/2449] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:42:56 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:42:56.548980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010e4d3-0544-457c-aa8c-90eef2d9b05b admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:56.560016 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010e4d3-0544-457c-aa8c-90eef2d9b05b admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:56.565460 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a71a1-baa8-4eab-bc5c-ea2197b45c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:56.566145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a71a1-baa8-4eab-bc5c-ea2197b45c39 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:56.566774 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a71a1-baa8-4eab-bc5c-ea2197b45c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:56.566902 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a71a1-baa8-4eab-bc5c-ea2197b45c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:56.567602 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a71a1-baa8-4eab-bc5c-ea2197b45c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:56.569237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a71a1-baa8-4eab-bc5c-ea2197b45c39 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:56.583927 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1231/2450] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:42:56.586006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22a802-3ad6-4a10-83d2-27aca44a5c2c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:56.596617 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22a802-3ad6-4a10-83d2-27aca44a5c2c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:56.601316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54989d31-1eb7-4958-a4d5-cceb0c4116aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:56.602212 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54989d31-1eb7-4958-a4d5-cceb0c4116aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:56.602493 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54989d31-1eb7-4958-a4d5-cceb0c4116aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:56.602779 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54989d31-1eb7-4958-a4d5-cceb0c4116aa admin admin] PATH_INFO: `/v3/projects/5f8d95e206894edeaeec5b78a980d866/users/48904324d7904afb8629bb5bf0e4a5c7/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:56.618559 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54989d31-1eb7-4958-a4d5-cceb0c4116aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f8d95e206894edeaeec5b78a980d866, user_id=48904324d7904afb8629bb5bf0e4a5c7, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:56.621754 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54989d31-1eb7-4958-a4d5-cceb0c4116aa admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:56.640215 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1220/2451] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:42:56 2022] PUT /identity/v3/projects/5f8d95e206894edeaeec5b78a980d866/users/48904324d7904afb8629bb5bf0e4a5c7/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:56.644440 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a71a1-baa8-4eab-bc5c-ea2197b45c39 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:56.654404 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a71a1-baa8-4eab-bc5c-ea2197b45c39 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:56.711538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc938662-8c79-4468-84d3-d57680e63616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:56.712212 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc938662-8c79-4468-84d3-d57680e63616 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:56.712495 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc938662-8c79-4468-84d3-d57680e63616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:56.712782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc938662-8c79-4468-84d3-d57680e63616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:56.713500 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc938662-8c79-4468-84d3-d57680e63616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:56.715167 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc938662-8c79-4468-84d3-d57680e63616 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:56.725651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1232/2452] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:42:56 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:42:56.734179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54989d31-1eb7-4958-a4d5-cceb0c4116aa admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:56.745508 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54989d31-1eb7-4958-a4d5-cceb0c4116aa admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:56.751391 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5782234f-13bd-4b6e-ac3d-e61b0f04b3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:56.752374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782234f-13bd-4b6e-ac3d-e61b0f04b3a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:56.752675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782234f-13bd-4b6e-ac3d-e61b0f04b3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:56.752970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782234f-13bd-4b6e-ac3d-e61b0f04b3a3 admin admin] PATH_INFO: `/v3/projects/5f8d95e206894edeaeec5b78a980d866/users/48904324d7904afb8629bb5bf0e4a5c7/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:56.759808 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5782234f-13bd-4b6e-ac3d-e61b0f04b3a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f8d95e206894edeaeec5b78a980d866, user_id=48904324d7904afb8629bb5bf0e4a5c7, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:56.763567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5782234f-13bd-4b6e-ac3d-e61b0f04b3a3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:56.775074 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1221/2453] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:42:56 2022] PUT /identity/v3/projects/5f8d95e206894edeaeec5b78a980d866/users/48904324d7904afb8629bb5bf0e4a5c7/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:56.786782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d69ab8-eeee-422c-8fc3-d44552fe8e5d None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:56.787265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d69ab8-eeee-422c-8fc3-d44552fe8e5d None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:56.787675 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d69ab8-eeee-422c-8fc3-d44552fe8e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:56.816015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-59d69ab8-eeee-422c-8fc3-d44552fe8e5d None None] MFA Rules not processed for user `48904324d7904afb8629bb5bf0e4a5c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:56.817163 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-59d69ab8-eeee-422c-8fc3-d44552fe8e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:56.889470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1233/2454] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:42:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:56.905846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5782234f-13bd-4b6e-ac3d-e61b0f04b3a3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:56.907774 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5782234f-13bd-4b6e-ac3d-e61b0f04b3a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:56.967434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5782234f-13bd-4b6e-ac3d-e61b0f04b3a3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:56.971075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94510bab-0815-49de-a694-84742de7efdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:56.971720 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94510bab-0815-49de-a694-84742de7efdb service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:56.972007 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94510bab-0815-49de-a694-84742de7efdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:56.972290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94510bab-0815-49de-a694-84742de7efdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:56.981442 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94510bab-0815-49de-a694-84742de7efdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:56.983182 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94510bab-0815-49de-a694-84742de7efdb service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:56.992963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1222/2455] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:42:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:42:57.021192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d69ab8-eeee-422c-8fc3-d44552fe8e5d None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:57.089143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:57.103813 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d69ab8-eeee-422c-8fc3-d44552fe8e5d None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:57.109737 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db5faa-69ab-4d77-8765-c1a01c22acbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:57.111605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db5faa-69ab-4d77-8765-c1a01c22acbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:57.111605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db5faa-69ab-4d77-8765-c1a01c22acbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:57.111605 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db5faa-69ab-4d77-8765-c1a01c22acbe admin admin] PATH_INFO: `/v3/users/ab22232eb33f4b278ac58058278db05d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:57.112685 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db5faa-69ab-4d77-8765-c1a01c22acbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab22232eb33f4b278ac58058278db05d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:57.114705 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85db5faa-69ab-4d77-8765-c1a01c22acbe admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:57.187560 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-85db5faa-69ab-4d77-8765-c1a01c22acbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab22232eb33f4b278ac58058278db05d'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:57.200880 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-85db5faa-69ab-4d77-8765-c1a01c22acbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab22232eb33f4b278ac58058278db05d'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:57.210246 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-85db5faa-69ab-4d77-8765-c1a01c22acbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab22232eb33f4b278ac58058278db05d'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:57.220336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-85db5faa-69ab-4d77-8765-c1a01c22acbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab22232eb33f4b278ac58058278db05d'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:57.221352 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-85db5faa-69ab-4d77-8765-c1a01c22acbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab22232eb33f4b278ac58058278db05d'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:57.228054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1234/2456] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:42:57 2022] DELETE /identity/v3/users/ab22232eb33f4b278ac58058278db05d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:57.233085 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9c682-66a6-4400-ab8f-b959428b9430 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:57.233582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9c682-66a6-4400-ab8f-b959428b9430 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:57.233756 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9c682-66a6-4400-ab8f-b959428b9430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:57.253261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-88f9c682-66a6-4400-ab8f-b959428b9430 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:57.254487 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-88f9c682-66a6-4400-ab8f-b959428b9430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:57.285977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:57.319869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1223/2457] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:57.329821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db5faa-69ab-4d77-8765-c1a01c22acbe admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:57.331349 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-85db5faa-69ab-4d77-8765-c1a01c22acbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:57.389151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db5faa-69ab-4d77-8765-c1a01c22acbe admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:57.392814 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8e24d-1cb0-4795-8d42-b7ab9bd7d093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:57.393469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8e24d-1cb0-4795-8d42-b7ab9bd7d093 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:57.393762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8e24d-1cb0-4795-8d42-b7ab9bd7d093 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:57.394044 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8e24d-1cb0-4795-8d42-b7ab9bd7d093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:57.403646 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8e24d-1cb0-4795-8d42-b7ab9bd7d093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:57.405094 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8e24d-1cb0-4795-8d42-b7ab9bd7d093 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:57.414785 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1235/2458] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:42:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:42:57.712700 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9c682-66a6-4400-ab8f-b959428b9430 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:57.714186 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-88f9c682-66a6-4400-ab8f-b959428b9430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:57.728931 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9c682-66a6-4400-ab8f-b959428b9430 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:57.734081 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a3541-18b2-44e4-9d30-797c63165ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:57.734977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a3541-18b2-44e4-9d30-797c63165ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:57.735263 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a3541-18b2-44e4-9d30-797c63165ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:57.735583 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a3541-18b2-44e4-9d30-797c63165ec1 admin admin] PATH_INFO: `/v3/projects/3a700048d1f44133bf1d76f5494db985` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:57.736796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a3541-18b2-44e4-9d30-797c63165ec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a700048d1f44133bf1d76f5494db985)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:57.738654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a3541-18b2-44e4-9d30-797c63165ec1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:57.789578 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3b6a3541-18b2-44e4-9d30-797c63165ec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a700048d1f44133bf1d76f5494db985', 'request_id': 'req-3b6a3541-18b2-44e4-9d30-797c63165ec1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:57.797619 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3b6a3541-18b2-44e4-9d30-797c63165ec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a700048d1f44133bf1d76f5494db985', 'request_id': 'req-3b6a3541-18b2-44e4-9d30-797c63165ec1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:57.803336 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3b6a3541-18b2-44e4-9d30-797c63165ec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a700048d1f44133bf1d76f5494db985', 'request_id': 'req-3b6a3541-18b2-44e4-9d30-797c63165ec1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:57.815176 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3b6a3541-18b2-44e4-9d30-797c63165ec1 admin admin] The token cache is being invalidate because project 3a700048d1f44133bf1d76f5494db985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:42:57.815472 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3b6a3541-18b2-44e4-9d30-797c63165ec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:57.816122 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3b6a3541-18b2-44e4-9d30-797c63165ec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:57.817664 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1224/2459] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:42:57 2022] DELETE /identity/v3/projects/3a700048d1f44133bf1d76f5494db985 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:57.821708 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8e24d-1cb0-4795-8d42-b7ab9bd7d093 service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:57.831598 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8e24d-1cb0-4795-8d42-b7ab9bd7d093 service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:57.889257 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770bfa73-7a1a-447e-b6a2-0b97fdeca6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:57.890285 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bfa73-7a1a-447e-b6a2-0b97fdeca6f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:57.890666 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bfa73-7a1a-447e-b6a2-0b97fdeca6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:57.891147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bfa73-7a1a-447e-b6a2-0b97fdeca6f8 admin admin] PATH_INFO: `/v3/users/48904324d7904afb8629bb5bf0e4a5c7` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:57.892587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bfa73-7a1a-447e-b6a2-0b97fdeca6f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48904324d7904afb8629bb5bf0e4a5c7)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:57.894727 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770bfa73-7a1a-447e-b6a2-0b97fdeca6f8 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:57.913885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:57.968936 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-770bfa73-7a1a-447e-b6a2-0b97fdeca6f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48904324d7904afb8629bb5bf0e4a5c7'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:57.983141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-770bfa73-7a1a-447e-b6a2-0b97fdeca6f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48904324d7904afb8629bb5bf0e4a5c7'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:57.990914 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-770bfa73-7a1a-447e-b6a2-0b97fdeca6f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48904324d7904afb8629bb5bf0e4a5c7'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:57.999680 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-770bfa73-7a1a-447e-b6a2-0b97fdeca6f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48904324d7904afb8629bb5bf0e4a5c7'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:58.000422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-770bfa73-7a1a-447e-b6a2-0b97fdeca6f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48904324d7904afb8629bb5bf0e4a5c7'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:58.007567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1236/2460] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:42:57 2022] DELETE /identity/v3/users/48904324d7904afb8629bb5bf0e4a5c7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:58.337709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a3541-18b2-44e4-9d30-797c63165ec1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:58.339423 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3b6a3541-18b2-44e4-9d30-797c63165ec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:58.357650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a3541-18b2-44e4-9d30-797c63165ec1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:58.442000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb163f19-73ee-4c33-b900-8f8496ecf681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:58.442872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb163f19-73ee-4c33-b900-8f8496ecf681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:58.443188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb163f19-73ee-4c33-b900-8f8496ecf681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:58.443467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb163f19-73ee-4c33-b900-8f8496ecf681 admin admin] PATH_INFO: `/v3/projects/5f8d95e206894edeaeec5b78a980d866` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:58.444710 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb163f19-73ee-4c33-b900-8f8496ecf681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f8d95e206894edeaeec5b78a980d866)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:58.446758 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb163f19-73ee-4c33-b900-8f8496ecf681 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:58.510299 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fb163f19-73ee-4c33-b900-8f8496ecf681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f8d95e206894edeaeec5b78a980d866', 'request_id': 'req-fb163f19-73ee-4c33-b900-8f8496ecf681'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:58.522074 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fb163f19-73ee-4c33-b900-8f8496ecf681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f8d95e206894edeaeec5b78a980d866', 'request_id': 'req-fb163f19-73ee-4c33-b900-8f8496ecf681'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:58.524591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:58.529994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fb163f19-73ee-4c33-b900-8f8496ecf681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f8d95e206894edeaeec5b78a980d866', 'request_id': 'req-fb163f19-73ee-4c33-b900-8f8496ecf681'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:58.549536 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fb163f19-73ee-4c33-b900-8f8496ecf681 admin admin] The token cache is being invalidate because project 5f8d95e206894edeaeec5b78a980d866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:42:58.550034 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fb163f19-73ee-4c33-b900-8f8496ecf681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:58.551522 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-fb163f19-73ee-4c33-b900-8f8496ecf681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:42:58.555017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1225/2461] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:42:58 2022] DELETE /identity/v3/projects/5f8d95e206894edeaeec5b78a980d866 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:58.561844 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f5eea-8fcc-4759-9dea-f6e93bf0471b None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:58.562161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f5eea-8fcc-4759-9dea-f6e93bf0471b None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:58.562564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f5eea-8fcc-4759-9dea-f6e93bf0471b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:58.585963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-e49f5eea-8fcc-4759-9dea-f6e93bf0471b None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:58.587214 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e49f5eea-8fcc-4759-9dea-f6e93bf0471b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:58.664872 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1237/2462] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:58.679479 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a494b9-8f54-4fbf-83a5-ac3942998ade None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:58.679999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a494b9-8f54-4fbf-83a5-ac3942998ade None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:58.680423 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a494b9-8f54-4fbf-83a5-ac3942998ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:58.701753 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-97a494b9-8f54-4fbf-83a5-ac3942998ade None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:58.703705 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-97a494b9-8f54-4fbf-83a5-ac3942998ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:58.712581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1226/2463] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:58.717484 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f5eea-8fcc-4759-9dea-f6e93bf0471b None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:58.727693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f5eea-8fcc-4759-9dea-f6e93bf0471b None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:58.732997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e71c69-aebb-42c4-ae46-7beea2a40765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:58.733773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e71c69-aebb-42c4-ae46-7beea2a40765 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:58.734057 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e71c69-aebb-42c4-ae46-7beea2a40765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:58.734397 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e71c69-aebb-42c4-ae46-7beea2a40765 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:58.735213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e71c69-aebb-42c4-ae46-7beea2a40765 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:42:58.735524 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e71c69-aebb-42c4-ae46-7beea2a40765 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:58.737191 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e71c69-aebb-42c4-ae46-7beea2a40765 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:58.751040 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1238/2464] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:42:58.755698 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a494b9-8f54-4fbf-83a5-ac3942998ade None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:58.765513 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a494b9-8f54-4fbf-83a5-ac3942998ade None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:58.771050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd228c-b0a2-467d-bbe9-0861da55883e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:58.771726 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd228c-b0a2-467d-bbe9-0861da55883e admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:58.772199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd228c-b0a2-467d-bbe9-0861da55883e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:58.772501 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd228c-b0a2-467d-bbe9-0861da55883e admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:58.773198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd228c-b0a2-467d-bbe9-0861da55883e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:58.775967 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd228c-b0a2-467d-bbe9-0861da55883e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:58.807969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1227/2465] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:42:58 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:42:58.814512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e71c69-aebb-42c4-ae46-7beea2a40765 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:58.825838 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e71c69-aebb-42c4-ae46-7beea2a40765 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:58.889819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01102fa4-1581-45bd-9bdc-851cb630cf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:58.890508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01102fa4-1581-45bd-9bdc-851cb630cf59 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:58.890792 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01102fa4-1581-45bd-9bdc-851cb630cf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:58.891108 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01102fa4-1581-45bd-9bdc-851cb630cf59 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:58.891806 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01102fa4-1581-45bd-9bdc-851cb630cf59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:58.893999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01102fa4-1581-45bd-9bdc-851cb630cf59 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:58.919345 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1239/2466] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:42:58 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:42:58.921481 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd228c-b0a2-467d-bbe9-0861da55883e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:58.934599 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd228c-b0a2-467d-bbe9-0861da55883e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:58.939695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77a483-d185-4f7d-9507-a4c353fabd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:58.940843 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77a483-d185-4f7d-9507-a4c353fabd8d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:58.941350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77a483-d185-4f7d-9507-a4c353fabd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:58.941862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77a483-d185-4f7d-9507-a4c353fabd8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:58.943183 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77a483-d185-4f7d-9507-a4c353fabd8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:58.946154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77a483-d185-4f7d-9507-a4c353fabd8d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:58.973200 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1228/2467] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:42:58.977343 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01102fa4-1581-45bd-9bdc-851cb630cf59 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:58.989971 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01102fa4-1581-45bd-9bdc-851cb630cf59 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:58.998972 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3d7e7-1e83-4d8b-b49a-22ea4347ec85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:59.001848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3d7e7-1e83-4d8b-b49a-22ea4347ec85 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:59.002164 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3d7e7-1e83-4d8b-b49a-22ea4347ec85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:59.002495 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3d7e7-1e83-4d8b-b49a-22ea4347ec85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:59.003407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3d7e7-1e83-4d8b-b49a-22ea4347ec85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:59.014194 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3d7e7-1e83-4d8b-b49a-22ea4347ec85 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:59.021439 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:59.028934 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1240/2468] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:42:59.035666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77a483-d185-4f7d-9507-a4c353fabd8d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:59.046835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77a483-d185-4f7d-9507-a4c353fabd8d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:59.052435 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb61d42-cbca-4db5-8f41-31827e6dbcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:59.053325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb61d42-cbca-4db5-8f41-31827e6dbcdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:59.053614 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb61d42-cbca-4db5-8f41-31827e6dbcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:59.053896 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb61d42-cbca-4db5-8f41-31827e6dbcdf admin admin] PATH_INFO: `/v3/projects/97cd15df895f4e7d8049745ffca63a54/users/30d71b97fcaa4fbbb2eb25902e290eab/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:59.069316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb61d42-cbca-4db5-8f41-31827e6dbcdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=97cd15df895f4e7d8049745ffca63a54, user_id=30d71b97fcaa4fbbb2eb25902e290eab, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:59.072826 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb61d42-cbca-4db5-8f41-31827e6dbcdf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:59.110246 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1229/2469] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:42:59 2022] PUT /identity/v3/projects/97cd15df895f4e7d8049745ffca63a54/users/30d71b97fcaa4fbbb2eb25902e290eab/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:59.123135 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97904b4a-2208-42a7-ab2a-49d8915bb85a None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:59.123628 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97904b4a-2208-42a7-ab2a-49d8915bb85a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:59.123940 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97904b4a-2208-42a7-ab2a-49d8915bb85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:59.167766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-97904b4a-2208-42a7-ab2a-49d8915bb85a None None] MFA Rules not processed for user `30d71b97fcaa4fbbb2eb25902e290eab`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:59.171773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-97904b4a-2208-42a7-ab2a-49d8915bb85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:59.264999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1241/2470] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:42:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:59.269621 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb61d42-cbca-4db5-8f41-31827e6dbcdf admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:59.359734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb61d42-cbca-4db5-8f41-31827e6dbcdf admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:59.365306 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ade321-d0b3-430f-98ce-d01437425f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:59.365977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ade321-d0b3-430f-98ce-d01437425f2e admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:59.366264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ade321-d0b3-430f-98ce-d01437425f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:59.366542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ade321-d0b3-430f-98ce-d01437425f2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:59.367279 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ade321-d0b3-430f-98ce-d01437425f2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:59.369389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ade321-d0b3-430f-98ce-d01437425f2e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:59.407088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1230/2471] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:42:59 2022] POST /identity/v3/projects => generated 387 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:42:59.408798 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97904b4a-2208-42a7-ab2a-49d8915bb85a None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:59.423170 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97904b4a-2208-42a7-ab2a-49d8915bb85a None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:59.502090 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892be49-3231-4d84-950f-47ea74cdb608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:59.502853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892be49-3231-4d84-950f-47ea74cdb608 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:59.503419 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892be49-3231-4d84-950f-47ea74cdb608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:59.503419 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892be49-3231-4d84-950f-47ea74cdb608 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:59.504308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892be49-3231-4d84-950f-47ea74cdb608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:59.506267 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892be49-3231-4d84-950f-47ea74cdb608 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:59.530321 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1242/2472] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:42:59 2022] POST /identity/v3/users => generated 300 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:42:59.535611 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ade321-d0b3-430f-98ce-d01437425f2e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:59.546389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ade321-d0b3-430f-98ce-d01437425f2e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:59.551378 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2d766-6729-4ba7-9eb4-360304dd191d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:59.552167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2d766-6729-4ba7-9eb4-360304dd191d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:59.552603 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2d766-6729-4ba7-9eb4-360304dd191d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:59.552925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2d766-6729-4ba7-9eb4-360304dd191d admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:59.553738 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2d766-6729-4ba7-9eb4-360304dd191d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:59.555486 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2d766-6729-4ba7-9eb4-360304dd191d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:59.573301 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1231/2473] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:42:59.574735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892be49-3231-4d84-950f-47ea74cdb608 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:59.586162 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892be49-3231-4d84-950f-47ea74cdb608 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:59.590955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46476cf-6eb7-48d3-a1da-10ae17df0891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:59.591914 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46476cf-6eb7-48d3-a1da-10ae17df0891 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:59.592192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46476cf-6eb7-48d3-a1da-10ae17df0891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:59.592452 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46476cf-6eb7-48d3-a1da-10ae17df0891 admin admin] PATH_INFO: `/v3/projects/a5faeff7b4db4160b154693fea91fce7/users/df5bf90ef93744e0b88ed876ac3244d9/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:59.615308 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46476cf-6eb7-48d3-a1da-10ae17df0891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5faeff7b4db4160b154693fea91fce7, user_id=df5bf90ef93744e0b88ed876ac3244d9, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:59.618011 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46476cf-6eb7-48d3-a1da-10ae17df0891 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:59.641783 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1243/2474] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:42:59 2022] PUT /identity/v3/projects/a5faeff7b4db4160b154693fea91fce7/users/df5bf90ef93744e0b88ed876ac3244d9/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:59.645984 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2d766-6729-4ba7-9eb4-360304dd191d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:59.657619 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2d766-6729-4ba7-9eb4-360304dd191d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:59.715758 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:59.722977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a265c68-200d-4208-a059-750762fbbddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:59.723669 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a265c68-200d-4208-a059-750762fbbddf admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:59.724298 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a265c68-200d-4208-a059-750762fbbddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:59.724926 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a265c68-200d-4208-a059-750762fbbddf admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:59.725862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a265c68-200d-4208-a059-750762fbbddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:59.728205 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a265c68-200d-4208-a059-750762fbbddf admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:59.740576 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1232/2475] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:42:59.744549 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46476cf-6eb7-48d3-a1da-10ae17df0891 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:59.755331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46476cf-6eb7-48d3-a1da-10ae17df0891 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:59.760261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939fb5d-fe6a-4ece-b2d6-7e40d4479d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:59.761163 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939fb5d-fe6a-4ece-b2d6-7e40d4479d93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:59.761447 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939fb5d-fe6a-4ece-b2d6-7e40d4479d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:59.761786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939fb5d-fe6a-4ece-b2d6-7e40d4479d93 admin admin] PATH_INFO: `/v3/projects/a5faeff7b4db4160b154693fea91fce7/users/df5bf90ef93744e0b88ed876ac3244d9/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:59.763995 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939fb5d-fe6a-4ece-b2d6-7e40d4479d93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5faeff7b4db4160b154693fea91fce7, user_id=df5bf90ef93744e0b88ed876ac3244d9, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:59.766629 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939fb5d-fe6a-4ece-b2d6-7e40d4479d93 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:59.778114 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1244/2476] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:42:59 2022] PUT /identity/v3/projects/a5faeff7b4db4160b154693fea91fce7/users/df5bf90ef93744e0b88ed876ac3244d9/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:42:59.788293 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13bf54-3810-4850-b665-0e2b144ec67b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:59.788592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13bf54-3810-4850-b665-0e2b144ec67b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:59.788884 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13bf54-3810-4850-b665-0e2b144ec67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:59.816188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-ae13bf54-3810-4850-b665-0e2b144ec67b None None] MFA Rules not processed for user `df5bf90ef93744e0b88ed876ac3244d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:42:59.817428 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ae13bf54-3810-4850-b665-0e2b144ec67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:59.886377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1233/2477] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:42:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:42:59.897825 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939fb5d-fe6a-4ece-b2d6-7e40d4479d93 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:42:59.899683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-6939fb5d-fe6a-4ece-b2d6-7e40d4479d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:42:59.953635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:42:59.966870 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939fb5d-fe6a-4ece-b2d6-7e40d4479d93 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:42:59.971691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ba200-caa8-4205-bbeb-0f1f1bdfca0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:42:59.972648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ba200-caa8-4205-bbeb-0f1f1bdfca0e service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:42:59.972933 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ba200-caa8-4205-bbeb-0f1f1bdfca0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:42:59.973260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ba200-caa8-4205-bbeb-0f1f1bdfca0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:42:59.983210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ba200-caa8-4205-bbeb-0f1f1bdfca0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:42:59.984663 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ba200-caa8-4205-bbeb-0f1f1bdfca0e service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:42:59.994125 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1245/2478] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:42:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:00.065207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13bf54-3810-4850-b665-0e2b144ec67b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:00.130116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13bf54-3810-4850-b665-0e2b144ec67b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:00.134737 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac163e5f-1118-4766-ac07-5ce98bcb9896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:00.135647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac163e5f-1118-4766-ac07-5ce98bcb9896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:00.135938 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac163e5f-1118-4766-ac07-5ce98bcb9896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:00.136222 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac163e5f-1118-4766-ac07-5ce98bcb9896 admin admin] PATH_INFO: `/v3/users/30d71b97fcaa4fbbb2eb25902e290eab` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:00.138275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac163e5f-1118-4766-ac07-5ce98bcb9896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30d71b97fcaa4fbbb2eb25902e290eab)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:00.140278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac163e5f-1118-4766-ac07-5ce98bcb9896 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:00.186732 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ac163e5f-1118-4766-ac07-5ce98bcb9896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30d71b97fcaa4fbbb2eb25902e290eab'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:00.195431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ac163e5f-1118-4766-ac07-5ce98bcb9896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30d71b97fcaa4fbbb2eb25902e290eab'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:00.203506 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ac163e5f-1118-4766-ac07-5ce98bcb9896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30d71b97fcaa4fbbb2eb25902e290eab'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:00.204119 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ac163e5f-1118-4766-ac07-5ce98bcb9896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30d71b97fcaa4fbbb2eb25902e290eab'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:00.204734 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ac163e5f-1118-4766-ac07-5ce98bcb9896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30d71b97fcaa4fbbb2eb25902e290eab'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:00.218396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:00.223001 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1234/2479] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:43:00 2022] DELETE /identity/v3/users/30d71b97fcaa4fbbb2eb25902e290eab => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:00.227730 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f691a-9c7b-4bdd-b9e7-697605afd866 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:00.227993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f691a-9c7b-4bdd-b9e7-697605afd866 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:00.228288 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f691a-9c7b-4bdd-b9e7-697605afd866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:00.245965 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-1b5f691a-9c7b-4bdd-b9e7-697605afd866 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:00.247259 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1b5f691a-9c7b-4bdd-b9e7-697605afd866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:00.306410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1246/2480] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:43:00.316204 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac163e5f-1118-4766-ac07-5ce98bcb9896 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:00.317599 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ac163e5f-1118-4766-ac07-5ce98bcb9896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:00.375642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac163e5f-1118-4766-ac07-5ce98bcb9896 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:00.380023 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c88b5-b2c6-42c0-b798-755e2204f324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:00.380688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c88b5-b2c6-42c0-b798-755e2204f324 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:00.380974 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c88b5-b2c6-42c0-b798-755e2204f324 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:00.381258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c88b5-b2c6-42c0-b798-755e2204f324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:00.393881 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c88b5-b2c6-42c0-b798-755e2204f324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:00.395481 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c88b5-b2c6-42c0-b798-755e2204f324 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:00.405728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1235/2481] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:00.667439 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f691a-9c7b-4bdd-b9e7-697605afd866 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:00.669047 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1b5f691a-9c7b-4bdd-b9e7-697605afd866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:00.682195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f691a-9c7b-4bdd-b9e7-697605afd866 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:00.687773 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43611f56-1156-4e52-9f01-dbdcf25e4def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:00.688817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43611f56-1156-4e52-9f01-dbdcf25e4def admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:00.689099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43611f56-1156-4e52-9f01-dbdcf25e4def admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:00.689380 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43611f56-1156-4e52-9f01-dbdcf25e4def admin admin] PATH_INFO: `/v3/projects/97cd15df895f4e7d8049745ffca63a54` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:00.690596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43611f56-1156-4e52-9f01-dbdcf25e4def admin admin] RBAC: Authorizing `identity:delete_project(project_id=97cd15df895f4e7d8049745ffca63a54)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:00.692572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43611f56-1156-4e52-9f01-dbdcf25e4def admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:00.739768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-43611f56-1156-4e52-9f01-dbdcf25e4def admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97cd15df895f4e7d8049745ffca63a54', 'request_id': 'req-43611f56-1156-4e52-9f01-dbdcf25e4def'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:00.745780 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-43611f56-1156-4e52-9f01-dbdcf25e4def admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97cd15df895f4e7d8049745ffca63a54', 'request_id': 'req-43611f56-1156-4e52-9f01-dbdcf25e4def'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:00.753294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-43611f56-1156-4e52-9f01-dbdcf25e4def admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97cd15df895f4e7d8049745ffca63a54', 'request_id': 'req-43611f56-1156-4e52-9f01-dbdcf25e4def'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:00.764250 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-43611f56-1156-4e52-9f01-dbdcf25e4def admin admin] The token cache is being invalidate because project 97cd15df895f4e7d8049745ffca63a54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:43:00.764540 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-43611f56-1156-4e52-9f01-dbdcf25e4def admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:00.765161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-43611f56-1156-4e52-9f01-dbdcf25e4def admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:00.766669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1247/2482] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:43:00 2022] DELETE /identity/v3/projects/97cd15df895f4e7d8049745ffca63a54 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:00.770621 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c88b5-b2c6-42c0-b798-755e2204f324 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:00.780345 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c88b5-b2c6-42c0-b798-755e2204f324 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:00.848328 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3848ea-f5a1-40e0-8a36-9b89c995de2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:00.849244 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3848ea-f5a1-40e0-8a36-9b89c995de2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:00.849534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3848ea-f5a1-40e0-8a36-9b89c995de2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:00.849849 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3848ea-f5a1-40e0-8a36-9b89c995de2f admin admin] PATH_INFO: `/v3/users/df5bf90ef93744e0b88ed876ac3244d9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:00.851088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3848ea-f5a1-40e0-8a36-9b89c995de2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=df5bf90ef93744e0b88ed876ac3244d9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:00.853060 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3848ea-f5a1-40e0-8a36-9b89c995de2f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:00.912763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ce3848ea-f5a1-40e0-8a36-9b89c995de2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df5bf90ef93744e0b88ed876ac3244d9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:00.920320 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ce3848ea-f5a1-40e0-8a36-9b89c995de2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df5bf90ef93744e0b88ed876ac3244d9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:00.929612 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ce3848ea-f5a1-40e0-8a36-9b89c995de2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df5bf90ef93744e0b88ed876ac3244d9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:00.931393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ce3848ea-f5a1-40e0-8a36-9b89c995de2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df5bf90ef93744e0b88ed876ac3244d9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:00.931956 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ce3848ea-f5a1-40e0-8a36-9b89c995de2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df5bf90ef93744e0b88ed876ac3244d9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:00.953696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1236/2483] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:43:00 2022] DELETE /identity/v3/users/df5bf90ef93744e0b88ed876ac3244d9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:43:01.227010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43611f56-1156-4e52-9f01-dbdcf25e4def admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:01.229217 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-43611f56-1156-4e52-9f01-dbdcf25e4def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:01.246126 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43611f56-1156-4e52-9f01-dbdcf25e4def admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:01.309673 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bd437-f60a-47a3-be0c-1378d92aa4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:01.310557 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bd437-f60a-47a3-be0c-1378d92aa4a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:01.310841 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bd437-f60a-47a3-be0c-1378d92aa4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:01.311167 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bd437-f60a-47a3-be0c-1378d92aa4a3 admin admin] PATH_INFO: `/v3/projects/a5faeff7b4db4160b154693fea91fce7` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:01.312360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bd437-f60a-47a3-be0c-1378d92aa4a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5faeff7b4db4160b154693fea91fce7)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:01.314297 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bd437-f60a-47a3-be0c-1378d92aa4a3 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:01.325159 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:01.367999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b57bd437-f60a-47a3-be0c-1378d92aa4a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5faeff7b4db4160b154693fea91fce7', 'request_id': 'req-b57bd437-f60a-47a3-be0c-1378d92aa4a3'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:01.374844 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b57bd437-f60a-47a3-be0c-1378d92aa4a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5faeff7b4db4160b154693fea91fce7', 'request_id': 'req-b57bd437-f60a-47a3-be0c-1378d92aa4a3'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:01.382783 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b57bd437-f60a-47a3-be0c-1378d92aa4a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5faeff7b4db4160b154693fea91fce7', 'request_id': 'req-b57bd437-f60a-47a3-be0c-1378d92aa4a3'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:01.393702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b57bd437-f60a-47a3-be0c-1378d92aa4a3 admin admin] The token cache is being invalidate because project a5faeff7b4db4160b154693fea91fce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:43:01.393995 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b57bd437-f60a-47a3-be0c-1378d92aa4a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:01.394647 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b57bd437-f60a-47a3-be0c-1378d92aa4a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:01.396200 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1248/2484] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:43:01 2022] DELETE /identity/v3/projects/a5faeff7b4db4160b154693fea91fce7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:01.400964 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d20b99-381e-4a24-b4f9-ce9824092cd6 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:01.401428 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d20b99-381e-4a24-b4f9-ce9824092cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:01.401524 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d20b99-381e-4a24-b4f9-ce9824092cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:01.419828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-83d20b99-381e-4a24-b4f9-ce9824092cd6 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:01.421047 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-83d20b99-381e-4a24-b4f9-ce9824092cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:01.434244 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:01.481716 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1237/2485] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:01.489514 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750ee44-f858-4485-929c-ac4c1d784215 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:01.489803 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750ee44-f858-4485-929c-ac4c1d784215 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:01.490084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750ee44-f858-4485-929c-ac4c1d784215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:01.508857 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-7750ee44-f858-4485-929c-ac4c1d784215 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:01.510067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7750ee44-f858-4485-929c-ac4c1d784215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:01.518131 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1249/2486] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:01.522411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d20b99-381e-4a24-b4f9-ce9824092cd6 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:01.531881 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d20b99-381e-4a24-b4f9-ce9824092cd6 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:01.536439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cbf61-b69b-419f-9195-14fd5f7ef801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:01.537090 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cbf61-b69b-419f-9195-14fd5f7ef801 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:01.537379 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cbf61-b69b-419f-9195-14fd5f7ef801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:01.537660 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cbf61-b69b-419f-9195-14fd5f7ef801 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:01.538378 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561cbf61-b69b-419f-9195-14fd5f7ef801 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:43:01.538691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561cbf61-b69b-419f-9195-14fd5f7ef801 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:01.540323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561cbf61-b69b-419f-9195-14fd5f7ef801 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:01.552318 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1238/2487] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:43:01.556376 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750ee44-f858-4485-929c-ac4c1d784215 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:01.565817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750ee44-f858-4485-929c-ac4c1d784215 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:01.570643 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a757da-7938-4101-bdf8-7ad22c1a11ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:01.571332 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a757da-7938-4101-bdf8-7ad22c1a11ed admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:01.571621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a757da-7938-4101-bdf8-7ad22c1a11ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:01.571906 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a757da-7938-4101-bdf8-7ad22c1a11ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:01.572602 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a757da-7938-4101-bdf8-7ad22c1a11ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:01.575384 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a757da-7938-4101-bdf8-7ad22c1a11ed admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:01.600982 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1250/2488] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:43:01 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:01.605122 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cbf61-b69b-419f-9195-14fd5f7ef801 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:01.615992 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cbf61-b69b-419f-9195-14fd5f7ef801 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:01.676196 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1b607-5870-46e8-8beb-76008060aaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:01.676870 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1b607-5870-46e8-8beb-76008060aaf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:01.677153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1b607-5870-46e8-8beb-76008060aaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:01.677435 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1b607-5870-46e8-8beb-76008060aaf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:01.678121 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1b607-5870-46e8-8beb-76008060aaf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:01.680324 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1b607-5870-46e8-8beb-76008060aaf7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:01.701460 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1239/2489] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:43:01 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:01.705709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a757da-7938-4101-bdf8-7ad22c1a11ed admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:01.716705 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a757da-7938-4101-bdf8-7ad22c1a11ed admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:01.721562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c14f59-4d3c-49ab-b7ee-be0575734fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:01.722329 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c14f59-4d3c-49ab-b7ee-be0575734fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:01.722684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c14f59-4d3c-49ab-b7ee-be0575734fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:01.723145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c14f59-4d3c-49ab-b7ee-be0575734fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:01.724369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c14f59-4d3c-49ab-b7ee-be0575734fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:01.725983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c14f59-4d3c-49ab-b7ee-be0575734fd2 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:01.737697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1251/2490] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:01.741809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1b607-5870-46e8-8beb-76008060aaf7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:01.752094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1b607-5870-46e8-8beb-76008060aaf7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:01.756863 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c47437-bef5-41df-8a26-a6dd3435cd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:01.757528 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c47437-bef5-41df-8a26-a6dd3435cd7e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:01.757822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c47437-bef5-41df-8a26-a6dd3435cd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:01.758112 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c47437-bef5-41df-8a26-a6dd3435cd7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:01.758844 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c47437-bef5-41df-8a26-a6dd3435cd7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:01.760474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c47437-bef5-41df-8a26-a6dd3435cd7e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:01.771439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1240/2491] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:01.775758 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c14f59-4d3c-49ab-b7ee-be0575734fd2 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:01.787114 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c14f59-4d3c-49ab-b7ee-be0575734fd2 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:01.792133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ca016-a0de-497f-9e81-ab8896552eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:01.793197 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ca016-a0de-497f-9e81-ab8896552eb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:01.793994 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ca016-a0de-497f-9e81-ab8896552eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:01.794282 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ca016-a0de-497f-9e81-ab8896552eb0 admin admin] PATH_INFO: `/v3/projects/f3c7899cccf647cbb187c08b26b977cb/users/ffc7bf1d951240acb6ed2e53b0c488cc/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:01.809821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ca016-a0de-497f-9e81-ab8896552eb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3c7899cccf647cbb187c08b26b977cb, user_id=ffc7bf1d951240acb6ed2e53b0c488cc, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:01.812551 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ca016-a0de-497f-9e81-ab8896552eb0 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:01.828223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1252/2492] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:01 2022] PUT /identity/v3/projects/f3c7899cccf647cbb187c08b26b977cb/users/ffc7bf1d951240acb6ed2e53b0c488cc/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:01.839522 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6ba45-0036-4de6-a6cc-5e4ddbd32815 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:01.839900 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6ba45-0036-4de6-a6cc-5e4ddbd32815 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:01.840817 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6ba45-0036-4de6-a6cc-5e4ddbd32815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:01.874676 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-a9c6ba45-0036-4de6-a6cc-5e4ddbd32815 None None] MFA Rules not processed for user `ffc7bf1d951240acb6ed2e53b0c488cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:01.875982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-a9c6ba45-0036-4de6-a6cc-5e4ddbd32815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:01.944982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1241/2493] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:01.950232 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ca016-a0de-497f-9e81-ab8896552eb0 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:02.018617 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ca016-a0de-497f-9e81-ab8896552eb0 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:02.023263 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0da9e-660f-481e-b9db-5699c256d167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:02.023923 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0da9e-660f-481e-b9db-5699c256d167 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:02.024208 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0da9e-660f-481e-b9db-5699c256d167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:02.024488 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0da9e-660f-481e-b9db-5699c256d167 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:02.025182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0da9e-660f-481e-b9db-5699c256d167 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:02.027148 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0da9e-660f-481e-b9db-5699c256d167 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:02.053320 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1253/2494] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:43:01 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:02.057361 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6ba45-0036-4de6-a6cc-5e4ddbd32815 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:02.068230 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6ba45-0036-4de6-a6cc-5e4ddbd32815 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:02.079749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:02.130856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a831d-61d4-4da0-892f-0e5f31e1eb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:02.131616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a831d-61d4-4da0-892f-0e5f31e1eb54 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:02.132114 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a831d-61d4-4da0-892f-0e5f31e1eb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:02.132413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a831d-61d4-4da0-892f-0e5f31e1eb54 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:02.133128 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a831d-61d4-4da0-892f-0e5f31e1eb54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:02.135149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a831d-61d4-4da0-892f-0e5f31e1eb54 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:02.157360 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1242/2495] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:43:02 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:02.161388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0da9e-660f-481e-b9db-5699c256d167 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:02.173000 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0da9e-660f-481e-b9db-5699c256d167 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:02.177641 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a841583-068c-4172-a4ce-9f9b4d97cf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:02.178305 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a841583-068c-4172-a4ce-9f9b4d97cf0e admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:02.178596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a841583-068c-4172-a4ce-9f9b4d97cf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:02.178884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a841583-068c-4172-a4ce-9f9b4d97cf0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:02.179646 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a841583-068c-4172-a4ce-9f9b4d97cf0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:02.181255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a841583-068c-4172-a4ce-9f9b4d97cf0e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:02.191702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1254/2496] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:02.195678 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a831d-61d4-4da0-892f-0e5f31e1eb54 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:02.206101 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a831d-61d4-4da0-892f-0e5f31e1eb54 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:02.210837 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ec2d6-ac3b-4c8a-b145-8408fd3b616e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:02.211750 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ec2d6-ac3b-4c8a-b145-8408fd3b616e admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:02.212033 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ec2d6-ac3b-4c8a-b145-8408fd3b616e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:02.212313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ec2d6-ac3b-4c8a-b145-8408fd3b616e admin admin] PATH_INFO: `/v3/projects/58fa20afaa1b4e39937cbf0a4450b898/users/fbc2a6abcd6743cfb20db679c498da2e/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:02.227344 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ec2d6-ac3b-4c8a-b145-8408fd3b616e admin admin] RBAC: Authorizing `identity:create_grant(project_id=58fa20afaa1b4e39937cbf0a4450b898, user_id=fbc2a6abcd6743cfb20db679c498da2e, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:02.229958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ec2d6-ac3b-4c8a-b145-8408fd3b616e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:02.243504 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1243/2497] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:02 2022] PUT /identity/v3/projects/58fa20afaa1b4e39937cbf0a4450b898/users/fbc2a6abcd6743cfb20db679c498da2e/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:43:02.247434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a841583-068c-4172-a4ce-9f9b4d97cf0e admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:02.257498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a841583-068c-4172-a4ce-9f9b4d97cf0e admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:02.262641 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:02.315967 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e2b11-7d62-438d-92c6-9ed08825f6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:02.316601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e2b11-7d62-438d-92c6-9ed08825f6f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:02.316892 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e2b11-7d62-438d-92c6-9ed08825f6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:02.317178 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e2b11-7d62-438d-92c6-9ed08825f6f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:02.317906 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e2b11-7d62-438d-92c6-9ed08825f6f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:02.319601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e2b11-7d62-438d-92c6-9ed08825f6f4 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:02.329844 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1255/2498] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:02.333979 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ec2d6-ac3b-4c8a-b145-8408fd3b616e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:02.349725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ec2d6-ac3b-4c8a-b145-8408fd3b616e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:02.354405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede2c2d-0dce-4245-be02-2398848b8149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:02.355353 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede2c2d-0dce-4245-be02-2398848b8149 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:02.355778 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede2c2d-0dce-4245-be02-2398848b8149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:02.356076 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede2c2d-0dce-4245-be02-2398848b8149 admin admin] PATH_INFO: `/v3/projects/58fa20afaa1b4e39937cbf0a4450b898/users/fbc2a6abcd6743cfb20db679c498da2e/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:02.358126 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede2c2d-0dce-4245-be02-2398848b8149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58fa20afaa1b4e39937cbf0a4450b898, user_id=fbc2a6abcd6743cfb20db679c498da2e, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:02.361874 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede2c2d-0dce-4245-be02-2398848b8149 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:02.374188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1244/2499] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:02 2022] PUT /identity/v3/projects/58fa20afaa1b4e39937cbf0a4450b898/users/fbc2a6abcd6743cfb20db679c498da2e/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:02.384690 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156aba6c-3315-46d7-b22c-6b804f87e854 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:02.384983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156aba6c-3315-46d7-b22c-6b804f87e854 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:02.385267 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156aba6c-3315-46d7-b22c-6b804f87e854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:02.412112 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-156aba6c-3315-46d7-b22c-6b804f87e854 None None] MFA Rules not processed for user `fbc2a6abcd6743cfb20db679c498da2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:02.413312 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-156aba6c-3315-46d7-b22c-6b804f87e854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:02.479872 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1256/2500] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:02.490412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede2c2d-0dce-4245-be02-2398848b8149 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:02.492223 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bede2c2d-0dce-4245-be02-2398848b8149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:02.551914 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede2c2d-0dce-4245-be02-2398848b8149 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:02.555577 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f1287-ee45-4492-bfbc-d3639670f887 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:02.556227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f1287-ee45-4492-bfbc-d3639670f887 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:02.556519 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f1287-ee45-4492-bfbc-d3639670f887 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:02.556808 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f1287-ee45-4492-bfbc-d3639670f887 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:02.568108 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f1287-ee45-4492-bfbc-d3639670f887 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:02.569495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f1287-ee45-4492-bfbc-d3639670f887 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:02.579223 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1245/2501] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:03.151797 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156aba6c-3315-46d7-b22c-6b804f87e854 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:03.218540 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156aba6c-3315-46d7-b22c-6b804f87e854 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:03.223031 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d75a58-35f4-4379-93ef-175ee6eb26c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:03.223900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d75a58-35f4-4379-93ef-175ee6eb26c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:03.224198 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d75a58-35f4-4379-93ef-175ee6eb26c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:03.224471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d75a58-35f4-4379-93ef-175ee6eb26c2 admin admin] PATH_INFO: `/v3/users/ffc7bf1d951240acb6ed2e53b0c488cc` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:03.225652 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d75a58-35f4-4379-93ef-175ee6eb26c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffc7bf1d951240acb6ed2e53b0c488cc)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:03.227592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d75a58-35f4-4379-93ef-175ee6eb26c2 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:03.260517 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:03.275640 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b5d75a58-35f4-4379-93ef-175ee6eb26c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffc7bf1d951240acb6ed2e53b0c488cc'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:03.288585 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b5d75a58-35f4-4379-93ef-175ee6eb26c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffc7bf1d951240acb6ed2e53b0c488cc'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:03.296365 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b5d75a58-35f4-4379-93ef-175ee6eb26c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffc7bf1d951240acb6ed2e53b0c488cc'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:03.303274 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b5d75a58-35f4-4379-93ef-175ee6eb26c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffc7bf1d951240acb6ed2e53b0c488cc'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:03.303898 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b5d75a58-35f4-4379-93ef-175ee6eb26c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffc7bf1d951240acb6ed2e53b0c488cc'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:03.311215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1257/2502] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:43:03 2022] DELETE /identity/v3/users/ffc7bf1d951240acb6ed2e53b0c488cc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:03.315809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f9d97-9632-4a8b-be85-8d29619d61c0 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:03.316132 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f9d97-9632-4a8b-be85-8d29619d61c0 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:03.316422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f9d97-9632-4a8b-be85-8d29619d61c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:03.334195 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-650f9d97-9632-4a8b-be85-8d29619d61c0 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:03.335370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-650f9d97-9632-4a8b-be85-8d29619d61c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:03.394327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1246/2503] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:03.403841 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d75a58-35f4-4379-93ef-175ee6eb26c2 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:03.405228 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b5d75a58-35f4-4379-93ef-175ee6eb26c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:03.461478 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d75a58-35f4-4379-93ef-175ee6eb26c2 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:03.465219 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe46f30-2ca1-4f51-b50c-1660f1f24a8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:03.465884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe46f30-2ca1-4f51-b50c-1660f1f24a8b service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:03.466146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe46f30-2ca1-4f51-b50c-1660f1f24a8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:03.466426 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe46f30-2ca1-4f51-b50c-1660f1f24a8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:03.475487 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe46f30-2ca1-4f51-b50c-1660f1f24a8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:03.476902 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe46f30-2ca1-4f51-b50c-1660f1f24a8b service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:03.486378 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1258/2504] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:03.752275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f9d97-9632-4a8b-be85-8d29619d61c0 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:03.753763 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-650f9d97-9632-4a8b-be85-8d29619d61c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:03.766257 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f9d97-9632-4a8b-be85-8d29619d61c0 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:03.770857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e028907-816f-494f-a57a-90f01588a921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:03.771761 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e028907-816f-494f-a57a-90f01588a921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:03.772052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e028907-816f-494f-a57a-90f01588a921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:03.772339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e028907-816f-494f-a57a-90f01588a921 admin admin] PATH_INFO: `/v3/projects/f3c7899cccf647cbb187c08b26b977cb` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:03.773566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e028907-816f-494f-a57a-90f01588a921 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3c7899cccf647cbb187c08b26b977cb)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:03.775460 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e028907-816f-494f-a57a-90f01588a921 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:03.782887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:03.823831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3e028907-816f-494f-a57a-90f01588a921 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3c7899cccf647cbb187c08b26b977cb', 'request_id': 'req-3e028907-816f-494f-a57a-90f01588a921'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:03.831561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3e028907-816f-494f-a57a-90f01588a921 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3c7899cccf647cbb187c08b26b977cb', 'request_id': 'req-3e028907-816f-494f-a57a-90f01588a921'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:03.837107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3e028907-816f-494f-a57a-90f01588a921 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3c7899cccf647cbb187c08b26b977cb', 'request_id': 'req-3e028907-816f-494f-a57a-90f01588a921'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:03.848505 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3e028907-816f-494f-a57a-90f01588a921 admin admin] The token cache is being invalidate because project f3c7899cccf647cbb187c08b26b977cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:43:03.848794 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3e028907-816f-494f-a57a-90f01588a921 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:03.849423 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-3e028907-816f-494f-a57a-90f01588a921 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:03.850966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1247/2505] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:43:03 2022] DELETE /identity/v3/projects/f3c7899cccf647cbb187c08b26b977cb => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:03.854863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe46f30-2ca1-4f51-b50c-1660f1f24a8b service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:03.864538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe46f30-2ca1-4f51-b50c-1660f1f24a8b service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:03.920767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f71fef-9721-4360-8750-584a12d48fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:03.921564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f71fef-9721-4360-8750-584a12d48fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:03.921851 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f71fef-9721-4360-8750-584a12d48fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:03.922132 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f71fef-9721-4360-8750-584a12d48fcf admin admin] PATH_INFO: `/v3/users/fbc2a6abcd6743cfb20db679c498da2e` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:03.923354 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f71fef-9721-4360-8750-584a12d48fcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbc2a6abcd6743cfb20db679c498da2e)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:03.925283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f71fef-9721-4360-8750-584a12d48fcf admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:03.971584 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-31f71fef-9721-4360-8750-584a12d48fcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbc2a6abcd6743cfb20db679c498da2e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:03.976671 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:03.987834 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-31f71fef-9721-4360-8750-584a12d48fcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbc2a6abcd6743cfb20db679c498da2e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:03.997649 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-31f71fef-9721-4360-8750-584a12d48fcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbc2a6abcd6743cfb20db679c498da2e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:04.006445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-31f71fef-9721-4360-8750-584a12d48fcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbc2a6abcd6743cfb20db679c498da2e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:04.007174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-31f71fef-9721-4360-8750-584a12d48fcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbc2a6abcd6743cfb20db679c498da2e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:04.013298 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1259/2506] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:43:03 2022] DELETE /identity/v3/users/fbc2a6abcd6743cfb20db679c498da2e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:04.473027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e028907-816f-494f-a57a-90f01588a921 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:04.474588 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3e028907-816f-494f-a57a-90f01588a921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:04.509480 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e028907-816f-494f-a57a-90f01588a921 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:04.573674 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5484172-b86c-426a-94c1-6daad0081d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:04.574839 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5484172-b86c-426a-94c1-6daad0081d74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:04.575331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5484172-b86c-426a-94c1-6daad0081d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:04.575620 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5484172-b86c-426a-94c1-6daad0081d74 admin admin] PATH_INFO: `/v3/projects/58fa20afaa1b4e39937cbf0a4450b898` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:04.576869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5484172-b86c-426a-94c1-6daad0081d74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58fa20afaa1b4e39937cbf0a4450b898)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:04.578785 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5484172-b86c-426a-94c1-6daad0081d74 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:04.628965 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c5484172-b86c-426a-94c1-6daad0081d74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58fa20afaa1b4e39937cbf0a4450b898', 'request_id': 'req-c5484172-b86c-426a-94c1-6daad0081d74'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:04.635966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c5484172-b86c-426a-94c1-6daad0081d74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58fa20afaa1b4e39937cbf0a4450b898', 'request_id': 'req-c5484172-b86c-426a-94c1-6daad0081d74'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:04.642964 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c5484172-b86c-426a-94c1-6daad0081d74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58fa20afaa1b4e39937cbf0a4450b898', 'request_id': 'req-c5484172-b86c-426a-94c1-6daad0081d74'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:04.654401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c5484172-b86c-426a-94c1-6daad0081d74 admin admin] The token cache is being invalidate because project 58fa20afaa1b4e39937cbf0a4450b898 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:43:04.654690 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c5484172-b86c-426a-94c1-6daad0081d74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:04.655354 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-c5484172-b86c-426a-94c1-6daad0081d74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:04.656884 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1248/2507] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:43:04 2022] DELETE /identity/v3/projects/58fa20afaa1b4e39937cbf0a4450b898 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:04.661874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09aec1-6720-4683-a6b4-f82fa30aa796 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:04.662352 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09aec1-6720-4683-a6b4-f82fa30aa796 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:04.662441 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09aec1-6720-4683-a6b4-f82fa30aa796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:04.681528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-ad09aec1-6720-4683-a6b4-f82fa30aa796 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:04.683026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-ad09aec1-6720-4683-a6b4-f82fa30aa796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:04.755540 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1260/2508] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:04.764474 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d793411-c588-44a6-aa4b-b7f708d27558 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:04.764994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d793411-c588-44a6-aa4b-b7f708d27558 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:04.765188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d793411-c588-44a6-aa4b-b7f708d27558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:04.783679 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5d793411-c588-44a6-aa4b-b7f708d27558 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:04.784992 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5d793411-c588-44a6-aa4b-b7f708d27558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:04.794062 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1249/2509] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:04.798591 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09aec1-6720-4683-a6b4-f82fa30aa796 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:04.808262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09aec1-6720-4683-a6b4-f82fa30aa796 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:04.813598 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ad65b-07ba-4e86-b08a-0374a6b85624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:04.814262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ad65b-07ba-4e86-b08a-0374a6b85624 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:04.814545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ad65b-07ba-4e86-b08a-0374a6b85624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:04.814825 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ad65b-07ba-4e86-b08a-0374a6b85624 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:04.815581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ad65b-07ba-4e86-b08a-0374a6b85624 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:43:04.815891 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ad65b-07ba-4e86-b08a-0374a6b85624 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:04.817494 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ad65b-07ba-4e86-b08a-0374a6b85624 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:04.829973 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1261/2510] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:43:04.834211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d793411-c588-44a6-aa4b-b7f708d27558 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:04.843733 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d793411-c588-44a6-aa4b-b7f708d27558 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:04.848402 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84255e-bf4e-4a34-be41-6356c93a991c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:04.849057 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84255e-bf4e-4a34-be41-6356c93a991c admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:04.849345 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84255e-bf4e-4a34-be41-6356c93a991c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:04.849627 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84255e-bf4e-4a34-be41-6356c93a991c admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:04.850331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84255e-bf4e-4a34-be41-6356c93a991c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:04.852276 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84255e-bf4e-4a34-be41-6356c93a991c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:04.867316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:04.886048 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1250/2511] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:43:04 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:04.887568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ad65b-07ba-4e86-b08a-0374a6b85624 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:04.897922 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ad65b-07ba-4e86-b08a-0374a6b85624 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:04.962619 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0635b7-cb9b-4f07-9469-7795ade03800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:04.963438 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0635b7-cb9b-4f07-9469-7795ade03800 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:04.963805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0635b7-cb9b-4f07-9469-7795ade03800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:04.964158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0635b7-cb9b-4f07-9469-7795ade03800 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:04.965606 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0635b7-cb9b-4f07-9469-7795ade03800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:04.967789 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0635b7-cb9b-4f07-9469-7795ade03800 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:05.002569 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1262/2512] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:43:04 2022] POST /identity/v3/users => generated 290 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:05.006150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84255e-bf4e-4a34-be41-6356c93a991c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:05.021788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84255e-bf4e-4a34-be41-6356c93a991c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:05.026314 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779ef32-9b37-46c2-ab22-f7aaa31ddcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:05.027017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779ef32-9b37-46c2-ab22-f7aaa31ddcc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:05.028020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779ef32-9b37-46c2-ab22-f7aaa31ddcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:05.028450 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779ef32-9b37-46c2-ab22-f7aaa31ddcc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:05.029461 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779ef32-9b37-46c2-ab22-f7aaa31ddcc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:05.030840 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779ef32-9b37-46c2-ab22-f7aaa31ddcc9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:05.044923 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1251/2513] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:05.048677 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0635b7-cb9b-4f07-9469-7795ade03800 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:05.061459 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0635b7-cb9b-4f07-9469-7795ade03800 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:05.079893 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873de916-79f5-4006-bace-b20f03fb5b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:05.081062 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873de916-79f5-4006-bace-b20f03fb5b08 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:05.081453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873de916-79f5-4006-bace-b20f03fb5b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:05.082145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873de916-79f5-4006-bace-b20f03fb5b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:05.084088 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873de916-79f5-4006-bace-b20f03fb5b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:05.086136 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873de916-79f5-4006-bace-b20f03fb5b08 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:05.098650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1263/2514] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:43:05.103551 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779ef32-9b37-46c2-ab22-f7aaa31ddcc9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:05.115080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779ef32-9b37-46c2-ab22-f7aaa31ddcc9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:05.119989 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13251fe1-0a23-4565-ae5f-a9bcd06afac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:05.120905 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13251fe1-0a23-4565-ae5f-a9bcd06afac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:05.126340 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13251fe1-0a23-4565-ae5f-a9bcd06afac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:05.126340 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13251fe1-0a23-4565-ae5f-a9bcd06afac3 admin admin] PATH_INFO: `/v3/projects/7a029a86d5964f91a0152ee7a6086f11/users/9df33249c9584b09a01b54dcf48c3c8a/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:05.151371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13251fe1-0a23-4565-ae5f-a9bcd06afac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a029a86d5964f91a0152ee7a6086f11, user_id=9df33249c9584b09a01b54dcf48c3c8a, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:05.155338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13251fe1-0a23-4565-ae5f-a9bcd06afac3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:05.173341 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1252/2515] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:05 2022] PUT /identity/v3/projects/7a029a86d5964f91a0152ee7a6086f11/users/9df33249c9584b09a01b54dcf48c3c8a/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:05.182726 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd33134-48c0-4199-a234-86711906a1a4 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:05.183048 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd33134-48c0-4199-a234-86711906a1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:05.183343 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd33134-48c0-4199-a234-86711906a1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:05.214185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-2fd33134-48c0-4199-a234-86711906a1a4 None None] MFA Rules not processed for user `9df33249c9584b09a01b54dcf48c3c8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:05.216156 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2fd33134-48c0-4199-a234-86711906a1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:05.244643 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:05.288292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1264/2516] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:05.292643 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13251fe1-0a23-4565-ae5f-a9bcd06afac3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:05.356705 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13251fe1-0a23-4565-ae5f-a9bcd06afac3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:05.361482 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a74b39-584f-496e-ac8f-8d116f2812fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:05.362555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a74b39-584f-496e-ac8f-8d116f2812fc admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:05.362555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a74b39-584f-496e-ac8f-8d116f2812fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:05.362789 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a74b39-584f-496e-ac8f-8d116f2812fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:05.363405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a74b39-584f-496e-ac8f-8d116f2812fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:05.365309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a74b39-584f-496e-ac8f-8d116f2812fc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:05.396675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1253/2517] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:43:05 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:43:05.400854 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd33134-48c0-4199-a234-86711906a1a4 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:05.416373 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd33134-48c0-4199-a234-86711906a1a4 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:05.474097 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66732e47-a6fe-4209-9c1e-c0edab02719b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:05.474781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66732e47-a6fe-4209-9c1e-c0edab02719b admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:05.475111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66732e47-a6fe-4209-9c1e-c0edab02719b admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:05.475897 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66732e47-a6fe-4209-9c1e-c0edab02719b admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:05.476108 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66732e47-a6fe-4209-9c1e-c0edab02719b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:05.479043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66732e47-a6fe-4209-9c1e-c0edab02719b admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:05.501518 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1265/2518] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:43:05 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:43:05.505542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a74b39-584f-496e-ac8f-8d116f2812fc admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:05.521788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a74b39-584f-496e-ac8f-8d116f2812fc admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:05.531260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c49c8-612a-4575-82dd-fb3cba79b831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:05.531260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c49c8-612a-4575-82dd-fb3cba79b831 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:05.531260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c49c8-612a-4575-82dd-fb3cba79b831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:05.531662 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c49c8-612a-4575-82dd-fb3cba79b831 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:05.532282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c49c8-612a-4575-82dd-fb3cba79b831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:05.534579 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c49c8-612a-4575-82dd-fb3cba79b831 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:05.552023 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1254/2519] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:05.556160 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66732e47-a6fe-4209-9c1e-c0edab02719b admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:05.568487 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66732e47-a6fe-4209-9c1e-c0edab02719b admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:05.574731 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b2450-9914-47fb-bb6c-cde13eed7810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:05.575596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b2450-9914-47fb-bb6c-cde13eed7810 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:05.575727 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b2450-9914-47fb-bb6c-cde13eed7810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:05.576068 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b2450-9914-47fb-bb6c-cde13eed7810 admin admin] PATH_INFO: `/v3/projects/e56cb81577554b799eeaab5208598370/users/cae22a3b60a94cad8119fe1de146cbf1/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:05.598812 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b2450-9914-47fb-bb6c-cde13eed7810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e56cb81577554b799eeaab5208598370, user_id=cae22a3b60a94cad8119fe1de146cbf1, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:05.603962 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b2450-9914-47fb-bb6c-cde13eed7810 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:05.628489 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1266/2520] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:05 2022] PUT /identity/v3/projects/e56cb81577554b799eeaab5208598370/users/cae22a3b60a94cad8119fe1de146cbf1/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:05.629681 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c49c8-612a-4575-82dd-fb3cba79b831 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:05.641277 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c49c8-612a-4575-82dd-fb3cba79b831 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:05.704973 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e95e6-6f09-4d92-ae1c-0403f242bd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:05.705669 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e95e6-6f09-4d92-ae1c-0403f242bd0e admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:05.705965 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e95e6-6f09-4d92-ae1c-0403f242bd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:05.706252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e95e6-6f09-4d92-ae1c-0403f242bd0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:05.707018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e95e6-6f09-4d92-ae1c-0403f242bd0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:05.708661 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e95e6-6f09-4d92-ae1c-0403f242bd0e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:05.719952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1255/2521] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:05.724067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b2450-9914-47fb-bb6c-cde13eed7810 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:05.736099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b2450-9914-47fb-bb6c-cde13eed7810 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:05.740816 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46755cb-8b17-40e8-a3ef-471551509c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:05.741716 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46755cb-8b17-40e8-a3ef-471551509c8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:05.742003 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46755cb-8b17-40e8-a3ef-471551509c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:05.742292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46755cb-8b17-40e8-a3ef-471551509c8a admin admin] PATH_INFO: `/v3/projects/e56cb81577554b799eeaab5208598370/users/cae22a3b60a94cad8119fe1de146cbf1/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:05.744331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46755cb-8b17-40e8-a3ef-471551509c8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e56cb81577554b799eeaab5208598370, user_id=cae22a3b60a94cad8119fe1de146cbf1, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:05.747842 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46755cb-8b17-40e8-a3ef-471551509c8a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:05.759350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1267/2522] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:05 2022] PUT /identity/v3/projects/e56cb81577554b799eeaab5208598370/users/cae22a3b60a94cad8119fe1de146cbf1/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:05.769976 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135c7c5-ce59-4e1c-9b27-2671078faead None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:05.770453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135c7c5-ce59-4e1c-9b27-2671078faead None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:05.770571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135c7c5-ce59-4e1c-9b27-2671078faead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:05.798557 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-c135c7c5-ce59-4e1c-9b27-2671078faead None None] MFA Rules not processed for user `cae22a3b60a94cad8119fe1de146cbf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:05.799798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-c135c7c5-ce59-4e1c-9b27-2671078faead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:05.820434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:05.875743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1256/2523] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:43:05.886241 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46755cb-8b17-40e8-a3ef-471551509c8a admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:05.888266 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b46755cb-8b17-40e8-a3ef-471551509c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:05.949763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46755cb-8b17-40e8-a3ef-471551509c8a admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:05.953535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f1fe4-e2d2-4044-90cd-69926bd8907c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:05.954191 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f1fe4-e2d2-4044-90cd-69926bd8907c service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:05.954481 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f1fe4-e2d2-4044-90cd-69926bd8907c service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:05.954763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f1fe4-e2d2-4044-90cd-69926bd8907c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:05.964163 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f1fe4-e2d2-4044-90cd-69926bd8907c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:05.965662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f1fe4-e2d2-4044-90cd-69926bd8907c service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:05.975366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1268/2524] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:06.338894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135c7c5-ce59-4e1c-9b27-2671078faead None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:06.400418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135c7c5-ce59-4e1c-9b27-2671078faead None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:06.406260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068de2d5-c415-4b3e-a3e5-ecf7a6431ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:06.407165 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068de2d5-c415-4b3e-a3e5-ecf7a6431ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:06.407454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068de2d5-c415-4b3e-a3e5-ecf7a6431ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:06.407737 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068de2d5-c415-4b3e-a3e5-ecf7a6431ccc admin admin] PATH_INFO: `/v3/users/9df33249c9584b09a01b54dcf48c3c8a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:06.408920 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068de2d5-c415-4b3e-a3e5-ecf7a6431ccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9df33249c9584b09a01b54dcf48c3c8a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:06.410840 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068de2d5-c415-4b3e-a3e5-ecf7a6431ccc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:06.458477 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-068de2d5-c415-4b3e-a3e5-ecf7a6431ccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9df33249c9584b09a01b54dcf48c3c8a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:06.466052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-068de2d5-c415-4b3e-a3e5-ecf7a6431ccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9df33249c9584b09a01b54dcf48c3c8a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:06.473570 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-068de2d5-c415-4b3e-a3e5-ecf7a6431ccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9df33249c9584b09a01b54dcf48c3c8a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:06.474240 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-068de2d5-c415-4b3e-a3e5-ecf7a6431ccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9df33249c9584b09a01b54dcf48c3c8a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:06.474867 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-068de2d5-c415-4b3e-a3e5-ecf7a6431ccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9df33249c9584b09a01b54dcf48c3c8a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:06.492778 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1257/2525] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:43:06 2022] DELETE /identity/v3/users/9df33249c9584b09a01b54dcf48c3c8a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:06.497573 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbe55c-e4ed-44ee-8fc4-2189c506334b None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:06.497952 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbe55c-e4ed-44ee-8fc4-2189c506334b None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:06.498246 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbe55c-e4ed-44ee-8fc4-2189c506334b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:06.515023 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-44cbe55c-e4ed-44ee-8fc4-2189c506334b None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:06.516216 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-44cbe55c-e4ed-44ee-8fc4-2189c506334b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:06.533230 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:06.576442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1269/2526] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:06.586041 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068de2d5-c415-4b3e-a3e5-ecf7a6431ccc admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:06.587538 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-068de2d5-c415-4b3e-a3e5-ecf7a6431ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:06.630604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:06.649731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068de2d5-c415-4b3e-a3e5-ecf7a6431ccc admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:06.654069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445493e-d6a7-4ecf-bc8a-48f37276f383 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:06.654754 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445493e-d6a7-4ecf-bc8a-48f37276f383 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:06.655077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445493e-d6a7-4ecf-bc8a-48f37276f383 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:06.655364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445493e-d6a7-4ecf-bc8a-48f37276f383 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:06.666950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445493e-d6a7-4ecf-bc8a-48f37276f383 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:06.666950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445493e-d6a7-4ecf-bc8a-48f37276f383 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:06.677581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1258/2527] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:06.972469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbe55c-e4ed-44ee-8fc4-2189c506334b None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:06.974048 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-44cbe55c-e4ed-44ee-8fc4-2189c506334b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:06.988508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbe55c-e4ed-44ee-8fc4-2189c506334b None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:06.993735 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b04db-635a-42f3-b3d1-fba872329780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:06.994648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b04db-635a-42f3-b3d1-fba872329780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:06.995017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b04db-635a-42f3-b3d1-fba872329780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:06.995323 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b04db-635a-42f3-b3d1-fba872329780 admin admin] PATH_INFO: `/v3/projects/7a029a86d5964f91a0152ee7a6086f11` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:06.997081 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b04db-635a-42f3-b3d1-fba872329780 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a029a86d5964f91a0152ee7a6086f11)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:06.999382 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b04db-635a-42f3-b3d1-fba872329780 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:07.048987 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-820b04db-635a-42f3-b3d1-fba872329780 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a029a86d5964f91a0152ee7a6086f11', 'request_id': 'req-820b04db-635a-42f3-b3d1-fba872329780'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:07.055155 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-820b04db-635a-42f3-b3d1-fba872329780 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a029a86d5964f91a0152ee7a6086f11', 'request_id': 'req-820b04db-635a-42f3-b3d1-fba872329780'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:07.064060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-820b04db-635a-42f3-b3d1-fba872329780 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a029a86d5964f91a0152ee7a6086f11', 'request_id': 'req-820b04db-635a-42f3-b3d1-fba872329780'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:07.077388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-820b04db-635a-42f3-b3d1-fba872329780 admin admin] The token cache is being invalidate because project 7a029a86d5964f91a0152ee7a6086f11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:43:07.077686 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-820b04db-635a-42f3-b3d1-fba872329780 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:07.078375 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-820b04db-635a-42f3-b3d1-fba872329780 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:07.080364 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1270/2528] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:43:06 2022] DELETE /identity/v3/projects/7a029a86d5964f91a0152ee7a6086f11 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:07.084419 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445493e-d6a7-4ecf-bc8a-48f37276f383 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:07.094635 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445493e-d6a7-4ecf-bc8a-48f37276f383 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:07.165363 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5d3ca-30df-4308-ad41-1fb18563ce5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:07.166508 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5d3ca-30df-4308-ad41-1fb18563ce5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:07.166797 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5d3ca-30df-4308-ad41-1fb18563ce5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:07.167115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5d3ca-30df-4308-ad41-1fb18563ce5d admin admin] PATH_INFO: `/v3/users/cae22a3b60a94cad8119fe1de146cbf1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:07.168865 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5d3ca-30df-4308-ad41-1fb18563ce5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cae22a3b60a94cad8119fe1de146cbf1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:07.170818 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5d3ca-30df-4308-ad41-1fb18563ce5d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:07.217850 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-48b5d3ca-30df-4308-ad41-1fb18563ce5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cae22a3b60a94cad8119fe1de146cbf1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:07.224845 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-48b5d3ca-30df-4308-ad41-1fb18563ce5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cae22a3b60a94cad8119fe1de146cbf1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:07.234877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-48b5d3ca-30df-4308-ad41-1fb18563ce5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cae22a3b60a94cad8119fe1de146cbf1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:07.235580 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-48b5d3ca-30df-4308-ad41-1fb18563ce5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cae22a3b60a94cad8119fe1de146cbf1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:07.236214 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-48b5d3ca-30df-4308-ad41-1fb18563ce5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cae22a3b60a94cad8119fe1de146cbf1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:07.253520 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1259/2529] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:43:07 2022] DELETE /identity/v3/users/cae22a3b60a94cad8119fe1de146cbf1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:07.531161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b04db-635a-42f3-b3d1-fba872329780 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:07.532871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-820b04db-635a-42f3-b3d1-fba872329780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:07.546476 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b04db-635a-42f3-b3d1-fba872329780 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:07.604436 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7668b-db2f-4bea-b496-f00838b84d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:07.605224 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7668b-db2f-4bea-b496-f00838b84d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:07.605627 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7668b-db2f-4bea-b496-f00838b84d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:07.606018 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7668b-db2f-4bea-b496-f00838b84d32 admin admin] PATH_INFO: `/v3/projects/e56cb81577554b799eeaab5208598370` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:07.607239 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7668b-db2f-4bea-b496-f00838b84d32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e56cb81577554b799eeaab5208598370)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:07.609181 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7668b-db2f-4bea-b496-f00838b84d32 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:07.656801 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:07.658099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-8bf7668b-db2f-4bea-b496-f00838b84d32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e56cb81577554b799eeaab5208598370', 'request_id': 'req-8bf7668b-db2f-4bea-b496-f00838b84d32'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:07.666868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-8bf7668b-db2f-4bea-b496-f00838b84d32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e56cb81577554b799eeaab5208598370', 'request_id': 'req-8bf7668b-db2f-4bea-b496-f00838b84d32'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:07.674496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-8bf7668b-db2f-4bea-b496-f00838b84d32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e56cb81577554b799eeaab5208598370', 'request_id': 'req-8bf7668b-db2f-4bea-b496-f00838b84d32'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:07.685267 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-8bf7668b-db2f-4bea-b496-f00838b84d32 admin admin] The token cache is being invalidate because project e56cb81577554b799eeaab5208598370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:43:07.685556 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-8bf7668b-db2f-4bea-b496-f00838b84d32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:07.686199 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-8bf7668b-db2f-4bea-b496-f00838b84d32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:07.687882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1271/2530] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:43:07 2022] DELETE /identity/v3/projects/e56cb81577554b799eeaab5208598370 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:07.693063 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891f6a5-2f06-4580-954f-cca6ffa4f689 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:07.693429 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891f6a5-2f06-4580-954f-cca6ffa4f689 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:07.693710 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891f6a5-2f06-4580-954f-cca6ffa4f689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:07.714368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-b891f6a5-2f06-4580-954f-cca6ffa4f689 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:07.716087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b891f6a5-2f06-4580-954f-cca6ffa4f689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:07.771466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:07.783301 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1260/2531] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:07.792317 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad632a-a407-414f-9ce1-bb0602183758 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:07.792610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad632a-a407-414f-9ce1-bb0602183758 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:07.793437 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad632a-a407-414f-9ce1-bb0602183758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:07.812578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-faad632a-a407-414f-9ce1-bb0602183758 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:07.813824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-faad632a-a407-414f-9ce1-bb0602183758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:07.821938 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1272/2532] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:07.826203 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891f6a5-2f06-4580-954f-cca6ffa4f689 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:07.835909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891f6a5-2f06-4580-954f-cca6ffa4f689 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:07.840551 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cc9a0-c886-44ca-83fc-1f19473e8703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:07.841200 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cc9a0-c886-44ca-83fc-1f19473e8703 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:07.841489 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cc9a0-c886-44ca-83fc-1f19473e8703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:07.841771 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cc9a0-c886-44ca-83fc-1f19473e8703 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:07.842494 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cc9a0-c886-44ca-83fc-1f19473e8703 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:43:07.842805 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cc9a0-c886-44ca-83fc-1f19473e8703 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:07.844440 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cc9a0-c886-44ca-83fc-1f19473e8703 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:07.856275 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1261/2533] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:43:07.860453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad632a-a407-414f-9ce1-bb0602183758 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:07.869666 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad632a-a407-414f-9ce1-bb0602183758 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:07.875041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a3f9e-849a-4883-9115-b7eac59bd3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:07.875706 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a3f9e-849a-4883-9115-b7eac59bd3e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:07.875996 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a3f9e-849a-4883-9115-b7eac59bd3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:07.876310 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a3f9e-849a-4883-9115-b7eac59bd3e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:07.877019 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a3f9e-849a-4883-9115-b7eac59bd3e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:07.878952 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a3f9e-849a-4883-9115-b7eac59bd3e8 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:07.903871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1273/2534] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:43:07 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:07.908066 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cc9a0-c886-44ca-83fc-1f19473e8703 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:07.918832 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cc9a0-c886-44ca-83fc-1f19473e8703 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:07.975828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481a8a7-1be9-4f49-88e2-2ee7cdd2ca86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:07.976538 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481a8a7-1be9-4f49-88e2-2ee7cdd2ca86 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:07.976829 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481a8a7-1be9-4f49-88e2-2ee7cdd2ca86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:07.977111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481a8a7-1be9-4f49-88e2-2ee7cdd2ca86 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:07.977846 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481a8a7-1be9-4f49-88e2-2ee7cdd2ca86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:07.979878 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481a8a7-1be9-4f49-88e2-2ee7cdd2ca86 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:08.005332 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1262/2535] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:43:07 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:43:08.006611 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a3f9e-849a-4883-9115-b7eac59bd3e8 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:08.017182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a3f9e-849a-4883-9115-b7eac59bd3e8 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:08.021935 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227a39e-8a89-4811-83c6-87448067711f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:08.022600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227a39e-8a89-4811-83c6-87448067711f admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:08.022888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227a39e-8a89-4811-83c6-87448067711f admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:08.023204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227a39e-8a89-4811-83c6-87448067711f admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:08.024055 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227a39e-8a89-4811-83c6-87448067711f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:08.025675 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227a39e-8a89-4811-83c6-87448067711f admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:08.036791 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1274/2536] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:08.040654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481a8a7-1be9-4f49-88e2-2ee7cdd2ca86 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:08.050913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481a8a7-1be9-4f49-88e2-2ee7cdd2ca86 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:08.055492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bca193-899d-4368-acad-9b7cad42344a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:08.056143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bca193-899d-4368-acad-9b7cad42344a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:08.056429 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bca193-899d-4368-acad-9b7cad42344a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:08.056708 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bca193-899d-4368-acad-9b7cad42344a admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:08.057427 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bca193-899d-4368-acad-9b7cad42344a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:08.059930 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bca193-899d-4368-acad-9b7cad42344a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:08.069745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1263/2537] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:08.073836 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227a39e-8a89-4811-83c6-87448067711f admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:08.084868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227a39e-8a89-4811-83c6-87448067711f admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:08.089313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f434e9a-696e-4ec8-a989-c41bdceab824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:08.090395 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f434e9a-696e-4ec8-a989-c41bdceab824 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:08.090681 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f434e9a-696e-4ec8-a989-c41bdceab824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:08.090992 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f434e9a-696e-4ec8-a989-c41bdceab824 admin admin] PATH_INFO: `/v3/projects/119f517b7e8a4ce182f785d70f111c82/users/30082d2dbe364ed18af37c7fffe2ce04/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:08.105123 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f434e9a-696e-4ec8-a989-c41bdceab824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=119f517b7e8a4ce182f785d70f111c82, user_id=30082d2dbe364ed18af37c7fffe2ce04, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:08.108061 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f434e9a-696e-4ec8-a989-c41bdceab824 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:08.120689 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1275/2538] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:08 2022] PUT /identity/v3/projects/119f517b7e8a4ce182f785d70f111c82/users/30082d2dbe364ed18af37c7fffe2ce04/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:08.131586 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b50624-b1fa-4045-bad6-58664b8db005 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:08.132009 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b50624-b1fa-4045-bad6-58664b8db005 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:08.132150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b50624-b1fa-4045-bad6-58664b8db005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:08.158401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-40b50624-b1fa-4045-bad6-58664b8db005 None None] MFA Rules not processed for user `30082d2dbe364ed18af37c7fffe2ce04`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:08.159650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-40b50624-b1fa-4045-bad6-58664b8db005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:08.226282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1264/2539] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:43:08.230861 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f434e9a-696e-4ec8-a989-c41bdceab824 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:08.292533 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f434e9a-696e-4ec8-a989-c41bdceab824 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:08.297713 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296fa6a6-9979-4994-a27c-bbda2a653469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:08.298366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fa6a6-9979-4994-a27c-bbda2a653469 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:08.298652 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fa6a6-9979-4994-a27c-bbda2a653469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:08.298947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fa6a6-9979-4994-a27c-bbda2a653469 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:08.299644 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296fa6a6-9979-4994-a27c-bbda2a653469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:08.301544 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296fa6a6-9979-4994-a27c-bbda2a653469 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:08.326547 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1276/2540] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:43:08 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:08.330606 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b50624-b1fa-4045-bad6-58664b8db005 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:08.341282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b50624-b1fa-4045-bad6-58664b8db005 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:08.389004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:08.399793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d25e95-28c9-416d-b5f6-21425e1fc40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:08.400478 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d25e95-28c9-416d-b5f6-21425e1fc40e admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:08.400774 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d25e95-28c9-416d-b5f6-21425e1fc40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:08.401059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d25e95-28c9-416d-b5f6-21425e1fc40e admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:08.401828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d25e95-28c9-416d-b5f6-21425e1fc40e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:08.404525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d25e95-28c9-416d-b5f6-21425e1fc40e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:08.427036 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1265/2541] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:43:08 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:08.428252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296fa6a6-9979-4994-a27c-bbda2a653469 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:08.438557 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296fa6a6-9979-4994-a27c-bbda2a653469 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:08.443298 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f290cbb-6e27-4a9f-bad2-5a136b6a245e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:08.443954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f290cbb-6e27-4a9f-bad2-5a136b6a245e admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:08.444237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f290cbb-6e27-4a9f-bad2-5a136b6a245e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:08.444513 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f290cbb-6e27-4a9f-bad2-5a136b6a245e admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:08.445236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f290cbb-6e27-4a9f-bad2-5a136b6a245e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:08.446820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f290cbb-6e27-4a9f-bad2-5a136b6a245e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:08.459601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1277/2542] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:08.463819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d25e95-28c9-416d-b5f6-21425e1fc40e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:08.474530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d25e95-28c9-416d-b5f6-21425e1fc40e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:08.479217 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765ebfc-81d3-4634-b054-3c7c068902f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:08.480104 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765ebfc-81d3-4634-b054-3c7c068902f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:08.480950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765ebfc-81d3-4634-b054-3c7c068902f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:08.481423 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765ebfc-81d3-4634-b054-3c7c068902f2 admin admin] PATH_INFO: `/v3/projects/42fa1ec52c2347f194ae70d2553d2c73/users/65a7f4b0f2064387a0ff24d727669980/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:08.497215 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765ebfc-81d3-4634-b054-3c7c068902f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42fa1ec52c2347f194ae70d2553d2c73, user_id=65a7f4b0f2064387a0ff24d727669980, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:08.499896 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765ebfc-81d3-4634-b054-3c7c068902f2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:08.516878 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1266/2543] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:08 2022] PUT /identity/v3/projects/42fa1ec52c2347f194ae70d2553d2c73/users/65a7f4b0f2064387a0ff24d727669980/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:08.520947 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f290cbb-6e27-4a9f-bad2-5a136b6a245e admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:08.530744 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f290cbb-6e27-4a9f-bad2-5a136b6a245e admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:08.578172 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:08.595365 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951f7cf-f9fd-4efe-a9c2-8ee46577a6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:08.596007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951f7cf-f9fd-4efe-a9c2-8ee46577a6c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:08.596374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951f7cf-f9fd-4efe-a9c2-8ee46577a6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:08.596685 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951f7cf-f9fd-4efe-a9c2-8ee46577a6c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:08.597548 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951f7cf-f9fd-4efe-a9c2-8ee46577a6c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:08.599300 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9951f7cf-f9fd-4efe-a9c2-8ee46577a6c4 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:08.613396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1278/2544] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:43:08.617620 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765ebfc-81d3-4634-b054-3c7c068902f2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:08.630554 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765ebfc-81d3-4634-b054-3c7c068902f2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:08.635859 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387f73b-bad5-41e0-aff6-627d6caa8ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:08.637446 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387f73b-bad5-41e0-aff6-627d6caa8ecc admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:08.637967 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387f73b-bad5-41e0-aff6-627d6caa8ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:08.638338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387f73b-bad5-41e0-aff6-627d6caa8ecc admin admin] PATH_INFO: `/v3/projects/42fa1ec52c2347f194ae70d2553d2c73/users/65a7f4b0f2064387a0ff24d727669980/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:08.640555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1387f73b-bad5-41e0-aff6-627d6caa8ecc admin admin] RBAC: Authorizing `identity:create_grant(project_id=42fa1ec52c2347f194ae70d2553d2c73, user_id=65a7f4b0f2064387a0ff24d727669980, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:08.643350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1387f73b-bad5-41e0-aff6-627d6caa8ecc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:08.659519 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1267/2545] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:08 2022] PUT /identity/v3/projects/42fa1ec52c2347f194ae70d2553d2c73/users/65a7f4b0f2064387a0ff24d727669980/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:43:08.670581 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e73fe-c9d9-469b-9038-7fab01bd17c1 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:08.670876 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e73fe-c9d9-469b-9038-7fab01bd17c1 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:08.671174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e73fe-c9d9-469b-9038-7fab01bd17c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:08.698290 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-a02e73fe-c9d9-469b-9038-7fab01bd17c1 None None] MFA Rules not processed for user `65a7f4b0f2064387a0ff24d727669980`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:08.699646 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-a02e73fe-c9d9-469b-9038-7fab01bd17c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:08.768564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1279/2546] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:08.780210 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387f73b-bad5-41e0-aff6-627d6caa8ecc admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:08.781961 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1387f73b-bad5-41e0-aff6-627d6caa8ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:08.845714 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387f73b-bad5-41e0-aff6-627d6caa8ecc admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:08.849403 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba332694-6841-44ba-aaca-d9fa877379ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:08.850054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba332694-6841-44ba-aaca-d9fa877379ed service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:08.850345 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba332694-6841-44ba-aaca-d9fa877379ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:08.850631 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba332694-6841-44ba-aaca-d9fa877379ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:08.860714 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba332694-6841-44ba-aaca-d9fa877379ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:08.862266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba332694-6841-44ba-aaca-d9fa877379ed service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:08.872917 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1268/2547] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:08.962489 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e73fe-c9d9-469b-9038-7fab01bd17c1 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:08.972774 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e73fe-c9d9-469b-9038-7fab01bd17c1 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:08.976337 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1b781-cb86-41a1-a729-229a9dd2ca09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:08.976982 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1b781-cb86-41a1-a729-229a9dd2ca09 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:08.977270 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1b781-cb86-41a1-a729-229a9dd2ca09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:08.977551 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1b781-cb86-41a1-a729-229a9dd2ca09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:09.033850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1b781-cb86-41a1-a729-229a9dd2ca09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:09.035334 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1b781-cb86-41a1-a729-229a9dd2ca09 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:09.045039 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1280/2548] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:09.302873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba332694-6841-44ba-aaca-d9fa877379ed service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:09.363864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba332694-6841-44ba-aaca-d9fa877379ed service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:09.368609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739b7e9-4763-4ad4-afc3-58870f535705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:09.369477 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739b7e9-4763-4ad4-afc3-58870f535705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:09.369777 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739b7e9-4763-4ad4-afc3-58870f535705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:09.370035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739b7e9-4763-4ad4-afc3-58870f535705 admin admin] PATH_INFO: `/v3/users/30082d2dbe364ed18af37c7fffe2ce04` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:09.371766 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739b7e9-4763-4ad4-afc3-58870f535705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30082d2dbe364ed18af37c7fffe2ce04)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:09.373674 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739b7e9-4763-4ad4-afc3-58870f535705 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:09.420169 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b739b7e9-4763-4ad4-afc3-58870f535705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30082d2dbe364ed18af37c7fffe2ce04'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:09.422155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:09.429896 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b739b7e9-4763-4ad4-afc3-58870f535705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30082d2dbe364ed18af37c7fffe2ce04'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:09.438538 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b739b7e9-4763-4ad4-afc3-58870f535705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30082d2dbe364ed18af37c7fffe2ce04'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:09.439204 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b739b7e9-4763-4ad4-afc3-58870f535705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30082d2dbe364ed18af37c7fffe2ce04'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:09.439819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b739b7e9-4763-4ad4-afc3-58870f535705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30082d2dbe364ed18af37c7fffe2ce04'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:09.458993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1269/2549] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:43:09 2022] DELETE /identity/v3/users/30082d2dbe364ed18af37c7fffe2ce04 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:09.463449 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96552d89-ad59-4f0e-bd1a-532276a4054d None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:09.463759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96552d89-ad59-4f0e-bd1a-532276a4054d None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:09.464042 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96552d89-ad59-4f0e-bd1a-532276a4054d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:09.481164 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-96552d89-ad59-4f0e-bd1a-532276a4054d None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:09.482381 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-96552d89-ad59-4f0e-bd1a-532276a4054d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:09.543777 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1281/2550] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:09.552592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739b7e9-4763-4ad4-afc3-58870f535705 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:09.554006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b739b7e9-4763-4ad4-afc3-58870f535705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:09.612409 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739b7e9-4763-4ad4-afc3-58870f535705 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:09.616181 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26567fad-25dd-4ecd-b2db-bf6c10d781ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:09.616838 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567fad-25dd-4ecd-b2db-bf6c10d781ca service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:09.617131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567fad-25dd-4ecd-b2db-bf6c10d781ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:09.617416 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26567fad-25dd-4ecd-b2db-bf6c10d781ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:09.627318 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26567fad-25dd-4ecd-b2db-bf6c10d781ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:09.628754 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26567fad-25dd-4ecd-b2db-bf6c10d781ca service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:09.638993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1270/2551] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:09.957667 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96552d89-ad59-4f0e-bd1a-532276a4054d None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:09.959394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-96552d89-ad59-4f0e-bd1a-532276a4054d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:09.977118 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96552d89-ad59-4f0e-bd1a-532276a4054d None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:09.983196 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869e27f-61f4-4e53-a91f-7090d34c5575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:09.985744 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869e27f-61f4-4e53-a91f-7090d34c5575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:09.985744 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869e27f-61f4-4e53-a91f-7090d34c5575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:09.986650 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869e27f-61f4-4e53-a91f-7090d34c5575 admin admin] PATH_INFO: `/v3/projects/119f517b7e8a4ce182f785d70f111c82` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:09.989063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869e27f-61f4-4e53-a91f-7090d34c5575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=119f517b7e8a4ce182f785d70f111c82)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:09.991173 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869e27f-61f4-4e53-a91f-7090d34c5575 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:09.994258 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:10.045147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-4869e27f-61f4-4e53-a91f-7090d34c5575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '119f517b7e8a4ce182f785d70f111c82', 'request_id': 'req-4869e27f-61f4-4e53-a91f-7090d34c5575'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:10.051973 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-4869e27f-61f4-4e53-a91f-7090d34c5575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '119f517b7e8a4ce182f785d70f111c82', 'request_id': 'req-4869e27f-61f4-4e53-a91f-7090d34c5575'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:10.061036 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-4869e27f-61f4-4e53-a91f-7090d34c5575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '119f517b7e8a4ce182f785d70f111c82', 'request_id': 'req-4869e27f-61f4-4e53-a91f-7090d34c5575'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:10.076498 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-4869e27f-61f4-4e53-a91f-7090d34c5575 admin admin] The token cache is being invalidate because project 119f517b7e8a4ce182f785d70f111c82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:43:10.076792 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-4869e27f-61f4-4e53-a91f-7090d34c5575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:10.077488 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-4869e27f-61f4-4e53-a91f-7090d34c5575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:10.079059 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1282/2552] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:43:09 2022] DELETE /identity/v3/projects/119f517b7e8a4ce182f785d70f111c82 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:10.083164 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26567fad-25dd-4ecd-b2db-bf6c10d781ca service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:10.093181 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26567fad-25dd-4ecd-b2db-bf6c10d781ca service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:10.155550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cf105-293b-47b9-987c-c982c3d5f580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:10.156412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cf105-293b-47b9-987c-c982c3d5f580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:10.156697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cf105-293b-47b9-987c-c982c3d5f580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:10.156978 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cf105-293b-47b9-987c-c982c3d5f580 admin admin] PATH_INFO: `/v3/users/65a7f4b0f2064387a0ff24d727669980` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:10.158152 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cf105-293b-47b9-987c-c982c3d5f580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65a7f4b0f2064387a0ff24d727669980)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:10.160079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cf105-293b-47b9-987c-c982c3d5f580 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:10.206021 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8e8cf105-293b-47b9-987c-c982c3d5f580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65a7f4b0f2064387a0ff24d727669980'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:10.212718 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8e8cf105-293b-47b9-987c-c982c3d5f580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65a7f4b0f2064387a0ff24d727669980'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:10.221224 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8e8cf105-293b-47b9-987c-c982c3d5f580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65a7f4b0f2064387a0ff24d727669980'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:10.221848 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8e8cf105-293b-47b9-987c-c982c3d5f580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65a7f4b0f2064387a0ff24d727669980'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:10.222465 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8e8cf105-293b-47b9-987c-c982c3d5f580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65a7f4b0f2064387a0ff24d727669980'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:10.229421 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:10.243553 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1271/2553] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:43:10 2022] DELETE /identity/v3/users/65a7f4b0f2064387a0ff24d727669980 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:10.516475 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869e27f-61f4-4e53-a91f-7090d34c5575 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:10.517919 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-4869e27f-61f4-4e53-a91f-7090d34c5575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:10.533229 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869e27f-61f4-4e53-a91f-7090d34c5575 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:10.589574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ea1e0-2ba4-40e2-881a-29bf2bf1a3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:10.590532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ea1e0-2ba4-40e2-881a-29bf2bf1a3a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:10.590827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ea1e0-2ba4-40e2-881a-29bf2bf1a3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:10.591139 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ea1e0-2ba4-40e2-881a-29bf2bf1a3a6 admin admin] PATH_INFO: `/v3/projects/42fa1ec52c2347f194ae70d2553d2c73` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:10.592316 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ea1e0-2ba4-40e2-881a-29bf2bf1a3a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42fa1ec52c2347f194ae70d2553d2c73)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:10.594536 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ea1e0-2ba4-40e2-881a-29bf2bf1a3a6 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:10.641727 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b52ea1e0-2ba4-40e2-881a-29bf2bf1a3a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42fa1ec52c2347f194ae70d2553d2c73', 'request_id': 'req-b52ea1e0-2ba4-40e2-881a-29bf2bf1a3a6'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:10.647826 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b52ea1e0-2ba4-40e2-881a-29bf2bf1a3a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42fa1ec52c2347f194ae70d2553d2c73', 'request_id': 'req-b52ea1e0-2ba4-40e2-881a-29bf2bf1a3a6'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:10.655262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b52ea1e0-2ba4-40e2-881a-29bf2bf1a3a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42fa1ec52c2347f194ae70d2553d2c73', 'request_id': 'req-b52ea1e0-2ba4-40e2-881a-29bf2bf1a3a6'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:10.665353 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b52ea1e0-2ba4-40e2-881a-29bf2bf1a3a6 admin admin] The token cache is being invalidate because project 42fa1ec52c2347f194ae70d2553d2c73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:43:10.665645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b52ea1e0-2ba4-40e2-881a-29bf2bf1a3a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:10.666534 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b52ea1e0-2ba4-40e2-881a-29bf2bf1a3a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:10.669093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1283/2554] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:43:10 2022] DELETE /identity/v3/projects/42fa1ec52c2347f194ae70d2553d2c73 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:43:11.110545 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cf105-293b-47b9-987c-c982c3d5f580 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:11.112006 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8e8cf105-293b-47b9-987c-c982c3d5f580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:11.169847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cf105-293b-47b9-987c-c982c3d5f580 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:11.216249 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda05599-8a25-4b23-b172-4597e1c22207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:11.216822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda05599-8a25-4b23-b172-4597e1c22207 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:11.217106 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda05599-8a25-4b23-b172-4597e1c22207 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:11.217386 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda05599-8a25-4b23-b172-4597e1c22207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:11.219173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-eda05599-8a25-4b23-b172-4597e1c22207 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:11.280278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda05599-8a25-4b23-b172-4597e1c22207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:11.281696 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda05599-8a25-4b23-b172-4597e1c22207 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:11.291426 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1272/2555] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:14.790330 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ea1e0-2ba4-40e2-881a-29bf2bf1a3a6 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:14.792696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b52ea1e0-2ba4-40e2-881a-29bf2bf1a3a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:14.816193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:14.857319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda05599-8a25-4b23-b172-4597e1c22207 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:14.858821 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-eda05599-8a25-4b23-b172-4597e1c22207 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:14.872180 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda05599-8a25-4b23-b172-4597e1c22207 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:14.874083 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ea1e0-2ba4-40e2-881a-29bf2bf1a3a6 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:14.876968 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a502ea-50cf-4a87-a923-9b1f033a5c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:14.877855 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a502ea-50cf-4a87-a923-9b1f033a5c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:14.878175 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a502ea-50cf-4a87-a923-9b1f033a5c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:14.878437 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a502ea-50cf-4a87-a923-9b1f033a5c49 admin admin] PATH_INFO: `/v3/users/03486c7630314392b85271455d12aba2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:14.879551 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5ace4-e339-484b-bb48-c9e8d4fc49d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101864afd59543e699189917391e6ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:14.879683 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a502ea-50cf-4a87-a923-9b1f033a5c49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03486c7630314392b85271455d12aba2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:14.880343 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5ace4-e339-484b-bb48-c9e8d4fc49d9 service placement] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:14.880637 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5ace4-e339-484b-bb48-c9e8d4fc49d9 service placement] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:14.880916 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5ace4-e339-484b-bb48-c9e8d4fc49d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:14.881584 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a502ea-50cf-4a87-a923-9b1f033a5c49 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:14.882829 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-53c5ace4-e339-484b-bb48-c9e8d4fc49d9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:14.897301 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5ace4-e339-484b-bb48-c9e8d4fc49d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:14.899816 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5ace4-e339-484b-bb48-c9e8d4fc49d9 service placement] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:14.915172 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1284/2556] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:14.941269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-66a502ea-50cf-4a87-a923-9b1f033a5c49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03486c7630314392b85271455d12aba2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:14.943349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:14.951824 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-66a502ea-50cf-4a87-a923-9b1f033a5c49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03486c7630314392b85271455d12aba2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:14.959521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-66a502ea-50cf-4a87-a923-9b1f033a5c49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03486c7630314392b85271455d12aba2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:14.960209 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-66a502ea-50cf-4a87-a923-9b1f033a5c49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03486c7630314392b85271455d12aba2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:14.960963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-66a502ea-50cf-4a87-a923-9b1f033a5c49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03486c7630314392b85271455d12aba2'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:14.978141 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1273/2557] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:43:14 2022] DELETE /identity/v3/users/03486c7630314392b85271455d12aba2 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:43:15.152240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5ace4-e339-484b-bb48-c9e8d4fc49d9 service placement] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:15.163667 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5ace4-e339-484b-bb48-c9e8d4fc49d9 service placement] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:15.221425 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d3f51-6891-4e04-87fb-6526d0d64926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:15.222321 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d3f51-6891-4e04-87fb-6526d0d64926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:15.222604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d3f51-6891-4e04-87fb-6526d0d64926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:15.222883 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d3f51-6891-4e04-87fb-6526d0d64926 admin admin] PATH_INFO: `/v3/projects/2735e40fe2564a6d8a5cdc7c32b0b869` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:15.224115 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d3f51-6891-4e04-87fb-6526d0d64926 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2735e40fe2564a6d8a5cdc7c32b0b869)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:15.226045 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d3f51-6891-4e04-87fb-6526d0d64926 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:15.271334 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ed0d3f51-6891-4e04-87fb-6526d0d64926 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2735e40fe2564a6d8a5cdc7c32b0b869', 'request_id': 'req-ed0d3f51-6891-4e04-87fb-6526d0d64926'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:15.277360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ed0d3f51-6891-4e04-87fb-6526d0d64926 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2735e40fe2564a6d8a5cdc7c32b0b869', 'request_id': 'req-ed0d3f51-6891-4e04-87fb-6526d0d64926'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:15.284695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ed0d3f51-6891-4e04-87fb-6526d0d64926 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2735e40fe2564a6d8a5cdc7c32b0b869', 'request_id': 'req-ed0d3f51-6891-4e04-87fb-6526d0d64926'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:15.295627 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ed0d3f51-6891-4e04-87fb-6526d0d64926 admin admin] The token cache is being invalidate because project 2735e40fe2564a6d8a5cdc7c32b0b869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:43:15.295917 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ed0d3f51-6891-4e04-87fb-6526d0d64926 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:15.296555 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ed0d3f51-6891-4e04-87fb-6526d0d64926 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:15.298084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1285/2558] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:43:15 2022] DELETE /identity/v3/projects/2735e40fe2564a6d8a5cdc7c32b0b869 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:15.303446 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a68d80-8425-4cd0-8589-4f9d68e8c82e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:15.303742 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a68d80-8425-4cd0-8589-4f9d68e8c82e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:15.304026 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a68d80-8425-4cd0-8589-4f9d68e8c82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:15.325941 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-83a68d80-8425-4cd0-8589-4f9d68e8c82e None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:15.327393 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-83a68d80-8425-4cd0-8589-4f9d68e8c82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:15.388859 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1274/2559] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:15.397222 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82affc-9cef-43b4-ae7a-b66881517913 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:15.397514 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82affc-9cef-43b4-ae7a-b66881517913 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:15.397800 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82affc-9cef-43b4-ae7a-b66881517913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:15.417716 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-8d82affc-9cef-43b4-ae7a-b66881517913 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:15.420169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8d82affc-9cef-43b4-ae7a-b66881517913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:15.428074 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1286/2560] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:15.432155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a68d80-8425-4cd0-8589-4f9d68e8c82e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:15.441874 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a68d80-8425-4cd0-8589-4f9d68e8c82e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:15.446308 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291365e-de28-4616-85bf-089c87f1939b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:15.447074 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291365e-de28-4616-85bf-089c87f1939b admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:15.447368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291365e-de28-4616-85bf-089c87f1939b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:15.447648 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291365e-de28-4616-85bf-089c87f1939b admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:15.448364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291365e-de28-4616-85bf-089c87f1939b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:43:15.448671 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291365e-de28-4616-85bf-089c87f1939b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:15.450490 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291365e-de28-4616-85bf-089c87f1939b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:15.461654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1275/2561] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:43:15.466781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82affc-9cef-43b4-ae7a-b66881517913 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:15.475931 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82affc-9cef-43b4-ae7a-b66881517913 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:15.481409 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171e0ea-4ce6-47ce-8421-a7f0e6000a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:15.482142 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171e0ea-4ce6-47ce-8421-a7f0e6000a89 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:15.482426 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171e0ea-4ce6-47ce-8421-a7f0e6000a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:15.482704 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171e0ea-4ce6-47ce-8421-a7f0e6000a89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:15.483496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171e0ea-4ce6-47ce-8421-a7f0e6000a89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:15.485395 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171e0ea-4ce6-47ce-8421-a7f0e6000a89 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:15.511169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1287/2562] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:43:15 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:43:15.514867 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291365e-de28-4616-85bf-089c87f1939b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:15.524815 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291365e-de28-4616-85bf-089c87f1939b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:15.580606 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17792d72-3ce8-4ede-91b1-a1c7b6fd93bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:15.581197 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17792d72-3ce8-4ede-91b1-a1c7b6fd93bb admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:15.581484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17792d72-3ce8-4ede-91b1-a1c7b6fd93bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:15.581762 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17792d72-3ce8-4ede-91b1-a1c7b6fd93bb admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:15.582451 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17792d72-3ce8-4ede-91b1-a1c7b6fd93bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:15.584371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17792d72-3ce8-4ede-91b1-a1c7b6fd93bb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:15.604332 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1276/2563] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:43:15 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:15.609205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171e0ea-4ce6-47ce-8421-a7f0e6000a89 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:15.619585 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171e0ea-4ce6-47ce-8421-a7f0e6000a89 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:15.624834 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd39985-4156-4f5c-8905-d3af20d7e5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:15.625499 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd39985-4156-4f5c-8905-d3af20d7e5cf admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:15.625791 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd39985-4156-4f5c-8905-d3af20d7e5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:15.626078 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd39985-4156-4f5c-8905-d3af20d7e5cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:15.626805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd39985-4156-4f5c-8905-d3af20d7e5cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:15.628535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd39985-4156-4f5c-8905-d3af20d7e5cf admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:15.633654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:15.639363 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1288/2564] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:15.643372 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17792d72-3ce8-4ede-91b1-a1c7b6fd93bb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:15.653825 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17792d72-3ce8-4ede-91b1-a1c7b6fd93bb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:15.658549 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a1f1d-5905-453b-b872-41c3692e4f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:15.659260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a1f1d-5905-453b-b872-41c3692e4f2c admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:15.659531 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a1f1d-5905-453b-b872-41c3692e4f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:15.659813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a1f1d-5905-453b-b872-41c3692e4f2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:15.660777 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a1f1d-5905-453b-b872-41c3692e4f2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:15.662352 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a1f1d-5905-453b-b872-41c3692e4f2c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:15.675605 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1277/2565] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:15.681562 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd39985-4156-4f5c-8905-d3af20d7e5cf admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:15.692133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd39985-4156-4f5c-8905-d3af20d7e5cf admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:15.696876 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c31bd-ccec-4bef-bb52-735049a377a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:15.697768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c31bd-ccec-4bef-bb52-735049a377a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:15.698038 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c31bd-ccec-4bef-bb52-735049a377a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:15.698361 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c31bd-ccec-4bef-bb52-735049a377a8 admin admin] PATH_INFO: `/v3/projects/de00d5152ffb48d1b596f92991d589a8/users/8ba03de1eb9a4513b51e236829acd205/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:15.713190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c31bd-ccec-4bef-bb52-735049a377a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de00d5152ffb48d1b596f92991d589a8, user_id=8ba03de1eb9a4513b51e236829acd205, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:15.716557 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c31bd-ccec-4bef-bb52-735049a377a8 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:15.729717 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1289/2566] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:15 2022] PUT /identity/v3/projects/de00d5152ffb48d1b596f92991d589a8/users/8ba03de1eb9a4513b51e236829acd205/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:15.740265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1798d7-c64f-4ec7-8e6a-a44466912449 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:15.740567 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1798d7-c64f-4ec7-8e6a-a44466912449 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:15.740855 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1798d7-c64f-4ec7-8e6a-a44466912449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:15.767442 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bd1798d7-c64f-4ec7-8e6a-a44466912449 None None] MFA Rules not processed for user `8ba03de1eb9a4513b51e236829acd205`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:15.768676 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bd1798d7-c64f-4ec7-8e6a-a44466912449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:15.789602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:15.839091 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1278/2567] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:15.844537 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c31bd-ccec-4bef-bb52-735049a377a8 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:15.908291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c31bd-ccec-4bef-bb52-735049a377a8 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:15.913138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410b5bf-9d25-4839-9a90-c5fc0ab17960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:15.913907 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410b5bf-9d25-4839-9a90-c5fc0ab17960 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:15.914269 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410b5bf-9d25-4839-9a90-c5fc0ab17960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:15.914598 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410b5bf-9d25-4839-9a90-c5fc0ab17960 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:15.915459 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410b5bf-9d25-4839-9a90-c5fc0ab17960 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:15.918849 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410b5bf-9d25-4839-9a90-c5fc0ab17960 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:15.945029 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1290/2568] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:43:15 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:15.949244 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1798d7-c64f-4ec7-8e6a-a44466912449 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:15.960047 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1798d7-c64f-4ec7-8e6a-a44466912449 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:16.016265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f9284-c6e8-4ba3-96f5-a35f78e76369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:16.016966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f9284-c6e8-4ba3-96f5-a35f78e76369 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:16.017254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f9284-c6e8-4ba3-96f5-a35f78e76369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:16.017537 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f9284-c6e8-4ba3-96f5-a35f78e76369 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:16.018265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f9284-c6e8-4ba3-96f5-a35f78e76369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:16.020201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f9284-c6e8-4ba3-96f5-a35f78e76369 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:16.039633 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1279/2569] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:43:15 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:16.044728 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410b5bf-9d25-4839-9a90-c5fc0ab17960 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:16.054870 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410b5bf-9d25-4839-9a90-c5fc0ab17960 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:16.060092 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c6b68-651f-4ac0-836b-95e440562fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:16.060759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c6b68-651f-4ac0-836b-95e440562fea admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:16.061045 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c6b68-651f-4ac0-836b-95e440562fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:16.061324 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c6b68-651f-4ac0-836b-95e440562fea admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:16.062052 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c6b68-651f-4ac0-836b-95e440562fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:16.063687 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c6b68-651f-4ac0-836b-95e440562fea admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:16.073936 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1291/2570] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:16.079055 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f9284-c6e8-4ba3-96f5-a35f78e76369 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:16.089303 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f9284-c6e8-4ba3-96f5-a35f78e76369 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:16.094395 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b8389-9c83-450b-b892-a52191a71134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:16.095325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b8389-9c83-450b-b892-a52191a71134 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:16.095618 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b8389-9c83-450b-b892-a52191a71134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:16.095904 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b8389-9c83-450b-b892-a52191a71134 admin admin] PATH_INFO: `/v3/projects/ae8deb8074014c20ae8e68fd045830b1/users/dc31e962ffa440b7be08ef35620f5c27/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:16.111420 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b8389-9c83-450b-b892-a52191a71134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae8deb8074014c20ae8e68fd045830b1, user_id=dc31e962ffa440b7be08ef35620f5c27, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:16.114203 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b8389-9c83-450b-b892-a52191a71134 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:16.129331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1280/2571] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:16 2022] PUT /identity/v3/projects/ae8deb8074014c20ae8e68fd045830b1/users/dc31e962ffa440b7be08ef35620f5c27/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:16.133281 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c6b68-651f-4ac0-836b-95e440562fea admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:16.143788 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c6b68-651f-4ac0-836b-95e440562fea admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:16.199251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863d02d-c489-4f5e-954b-e6ef2d3868b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:16.199925 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863d02d-c489-4f5e-954b-e6ef2d3868b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:16.200210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863d02d-c489-4f5e-954b-e6ef2d3868b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:16.200491 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863d02d-c489-4f5e-954b-e6ef2d3868b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:16.201207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863d02d-c489-4f5e-954b-e6ef2d3868b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:16.202782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a863d02d-c489-4f5e-954b-e6ef2d3868b4 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:16.212369 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1292/2572] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:16.217093 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b8389-9c83-450b-b892-a52191a71134 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:16.227305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b8389-9c83-450b-b892-a52191a71134 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:16.233991 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12baddeb-8b68-4bc3-8664-e99825f58c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:16.234874 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12baddeb-8b68-4bc3-8664-e99825f58c47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:16.235195 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12baddeb-8b68-4bc3-8664-e99825f58c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:16.235481 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12baddeb-8b68-4bc3-8664-e99825f58c47 admin admin] PATH_INFO: `/v3/projects/ae8deb8074014c20ae8e68fd045830b1/users/dc31e962ffa440b7be08ef35620f5c27/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:16.237452 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12baddeb-8b68-4bc3-8664-e99825f58c47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae8deb8074014c20ae8e68fd045830b1, user_id=dc31e962ffa440b7be08ef35620f5c27, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:16.240077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12baddeb-8b68-4bc3-8664-e99825f58c47 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:16.250059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1281/2573] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:16 2022] PUT /identity/v3/projects/ae8deb8074014c20ae8e68fd045830b1/users/dc31e962ffa440b7be08ef35620f5c27/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:16.267280 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd0ef4-46c4-4550-b730-47ebd1af7974 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:16.267590 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd0ef4-46c4-4550-b730-47ebd1af7974 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:16.267865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd0ef4-46c4-4550-b730-47ebd1af7974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:16.295324 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-8afd0ef4-46c4-4550-b730-47ebd1af7974 None None] MFA Rules not processed for user `dc31e962ffa440b7be08ef35620f5c27`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:16.296535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8afd0ef4-46c4-4550-b730-47ebd1af7974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:16.329529 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:16.366235 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1293/2574] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:16.377600 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12baddeb-8b68-4bc3-8664-e99825f58c47 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:16.379366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-12baddeb-8b68-4bc3-8664-e99825f58c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:16.437236 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12baddeb-8b68-4bc3-8664-e99825f58c47 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:16.440918 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09195ba0-4822-4e4b-a231-6959125be95a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:16.441565 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09195ba0-4822-4e4b-a231-6959125be95a service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:16.441856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09195ba0-4822-4e4b-a231-6959125be95a service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:16.442144 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09195ba0-4822-4e4b-a231-6959125be95a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:16.451762 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09195ba0-4822-4e4b-a231-6959125be95a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:16.453206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09195ba0-4822-4e4b-a231-6959125be95a service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:16.463021 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1282/2575] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:16.533204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd0ef4-46c4-4550-b730-47ebd1af7974 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:16.543367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd0ef4-46c4-4550-b730-47ebd1af7974 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:16.547025 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bcc46-03a9-4690-bde1-cfde494450bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:16.547671 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bcc46-03a9-4690-bde1-cfde494450bf service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:16.547960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bcc46-03a9-4690-bde1-cfde494450bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:16.548244 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bcc46-03a9-4690-bde1-cfde494450bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:16.604814 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bcc46-03a9-4690-bde1-cfde494450bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:16.606291 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bcc46-03a9-4690-bde1-cfde494450bf service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:16.615557 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1294/2576] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:26.458569 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:26.464451 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09195ba0-4822-4e4b-a231-6959125be95a service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:26.528717 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09195ba0-4822-4e4b-a231-6959125be95a service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:26.533344 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96959925-0c03-45da-8ca4-ff4d630a229c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:26.534210 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96959925-0c03-45da-8ca4-ff4d630a229c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:26.534501 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96959925-0c03-45da-8ca4-ff4d630a229c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:26.534781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96959925-0c03-45da-8ca4-ff4d630a229c admin admin] PATH_INFO: `/v3/users/8ba03de1eb9a4513b51e236829acd205` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:26.536004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96959925-0c03-45da-8ca4-ff4d630a229c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ba03de1eb9a4513b51e236829acd205)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:26.537906 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96959925-0c03-45da-8ca4-ff4d630a229c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:26.586826 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96959925-0c03-45da-8ca4-ff4d630a229c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ba03de1eb9a4513b51e236829acd205'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:26.595512 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96959925-0c03-45da-8ca4-ff4d630a229c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ba03de1eb9a4513b51e236829acd205'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:26.603990 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96959925-0c03-45da-8ca4-ff4d630a229c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ba03de1eb9a4513b51e236829acd205'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:26.604612 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96959925-0c03-45da-8ca4-ff4d630a229c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ba03de1eb9a4513b51e236829acd205'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:26.605272 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-96959925-0c03-45da-8ca4-ff4d630a229c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ba03de1eb9a4513b51e236829acd205'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:26.611506 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=41 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:26.623822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1283/2577] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:43:26 2022] DELETE /identity/v3/users/8ba03de1eb9a4513b51e236829acd205 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:26.628233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab31b0a-7cde-497f-98b6-e7e3c2d1df33 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:26.628523 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab31b0a-7cde-497f-98b6-e7e3c2d1df33 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:26.628804 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab31b0a-7cde-497f-98b6-e7e3c2d1df33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:26.645975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-dab31b0a-7cde-497f-98b6-e7e3c2d1df33 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:26.647197 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-dab31b0a-7cde-497f-98b6-e7e3c2d1df33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:26.705318 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1295/2578] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:43:26.714893 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96959925-0c03-45da-8ca4-ff4d630a229c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:26.716821 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-96959925-0c03-45da-8ca4-ff4d630a229c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:26.776935 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96959925-0c03-45da-8ca4-ff4d630a229c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:26.781152 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ccf22-0ad6-405e-8f7d-820b7cbd9569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:26.781806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ccf22-0ad6-405e-8f7d-820b7cbd9569 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:26.782095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ccf22-0ad6-405e-8f7d-820b7cbd9569 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:26.782380 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ccf22-0ad6-405e-8f7d-820b7cbd9569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:26.791616 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ccf22-0ad6-405e-8f7d-820b7cbd9569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:26.793913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ccf22-0ad6-405e-8f7d-820b7cbd9569 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:26.803395 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1284/2579] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:27.073259 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab31b0a-7cde-497f-98b6-e7e3c2d1df33 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:27.075231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-dab31b0a-7cde-497f-98b6-e7e3c2d1df33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:27.090160 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab31b0a-7cde-497f-98b6-e7e3c2d1df33 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:27.097259 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ba9e6-1649-4c5d-9379-1dc5a71b0eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:27.098082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ba9e6-1649-4c5d-9379-1dc5a71b0eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:27.098412 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ba9e6-1649-4c5d-9379-1dc5a71b0eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:27.098725 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ba9e6-1649-4c5d-9379-1dc5a71b0eac admin admin] PATH_INFO: `/v3/projects/de00d5152ffb48d1b596f92991d589a8` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:27.099899 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ba9e6-1649-4c5d-9379-1dc5a71b0eac admin admin] RBAC: Authorizing `identity:delete_project(project_id=de00d5152ffb48d1b596f92991d589a8)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:27.102625 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ba9e6-1649-4c5d-9379-1dc5a71b0eac admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:27.154357 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ff1ba9e6-1649-4c5d-9379-1dc5a71b0eac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de00d5152ffb48d1b596f92991d589a8', 'request_id': 'req-ff1ba9e6-1649-4c5d-9379-1dc5a71b0eac'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:27.161432 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ff1ba9e6-1649-4c5d-9379-1dc5a71b0eac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de00d5152ffb48d1b596f92991d589a8', 'request_id': 'req-ff1ba9e6-1649-4c5d-9379-1dc5a71b0eac'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:27.169719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ff1ba9e6-1649-4c5d-9379-1dc5a71b0eac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de00d5152ffb48d1b596f92991d589a8', 'request_id': 'req-ff1ba9e6-1649-4c5d-9379-1dc5a71b0eac'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:27.180125 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ff1ba9e6-1649-4c5d-9379-1dc5a71b0eac admin admin] The token cache is being invalidate because project de00d5152ffb48d1b596f92991d589a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:43:27.180420 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ff1ba9e6-1649-4c5d-9379-1dc5a71b0eac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:27.181049 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-ff1ba9e6-1649-4c5d-9379-1dc5a71b0eac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:27.182571 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1296/2580] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:43:27 2022] DELETE /identity/v3/projects/de00d5152ffb48d1b596f92991d589a8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:27.187925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ccf22-0ad6-405e-8f7d-820b7cbd9569 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:27.198118 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ccf22-0ad6-405e-8f7d-820b7cbd9569 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:27.217309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:27.258283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3af1d-8e15-42a9-9d4f-0ac775037b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:27.259255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3af1d-8e15-42a9-9d4f-0ac775037b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:27.259541 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3af1d-8e15-42a9-9d4f-0ac775037b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:27.259818 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3af1d-8e15-42a9-9d4f-0ac775037b61 admin admin] PATH_INFO: `/v3/users/dc31e962ffa440b7be08ef35620f5c27` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:27.260994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3af1d-8e15-42a9-9d4f-0ac775037b61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc31e962ffa440b7be08ef35620f5c27)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:27.262996 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3af1d-8e15-42a9-9d4f-0ac775037b61 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:27.309902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b3c3af1d-8e15-42a9-9d4f-0ac775037b61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc31e962ffa440b7be08ef35620f5c27'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:27.316885 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b3c3af1d-8e15-42a9-9d4f-0ac775037b61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc31e962ffa440b7be08ef35620f5c27'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:27.324975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b3c3af1d-8e15-42a9-9d4f-0ac775037b61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc31e962ffa440b7be08ef35620f5c27'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:27.325596 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b3c3af1d-8e15-42a9-9d4f-0ac775037b61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc31e962ffa440b7be08ef35620f5c27'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:27.326217 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-b3c3af1d-8e15-42a9-9d4f-0ac775037b61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc31e962ffa440b7be08ef35620f5c27'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:27.343520 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1285/2581] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:43:27 2022] DELETE /identity/v3/users/dc31e962ffa440b7be08ef35620f5c27 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:27.621050 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ba9e6-1649-4c5d-9379-1dc5a71b0eac admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:27.622518 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-ff1ba9e6-1649-4c5d-9379-1dc5a71b0eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:27.646675 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ba9e6-1649-4c5d-9379-1dc5a71b0eac admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:27.705707 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee26154-aa94-4743-b486-11dc225d86d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:27.706508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee26154-aa94-4743-b486-11dc225d86d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:27.706798 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee26154-aa94-4743-b486-11dc225d86d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:27.707195 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee26154-aa94-4743-b486-11dc225d86d1 admin admin] PATH_INFO: `/v3/projects/ae8deb8074014c20ae8e68fd045830b1` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:27.708378 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee26154-aa94-4743-b486-11dc225d86d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae8deb8074014c20ae8e68fd045830b1)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:27.710283 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee26154-aa94-4743-b486-11dc225d86d1 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:27.740388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:27.757511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-5ee26154-aa94-4743-b486-11dc225d86d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae8deb8074014c20ae8e68fd045830b1', 'request_id': 'req-5ee26154-aa94-4743-b486-11dc225d86d1'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:27.764065 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-5ee26154-aa94-4743-b486-11dc225d86d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae8deb8074014c20ae8e68fd045830b1', 'request_id': 'req-5ee26154-aa94-4743-b486-11dc225d86d1'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:27.771617 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-5ee26154-aa94-4743-b486-11dc225d86d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae8deb8074014c20ae8e68fd045830b1', 'request_id': 'req-5ee26154-aa94-4743-b486-11dc225d86d1'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:27.782603 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-5ee26154-aa94-4743-b486-11dc225d86d1 admin admin] The token cache is being invalidate because project ae8deb8074014c20ae8e68fd045830b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:43:27.782975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-5ee26154-aa94-4743-b486-11dc225d86d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:27.783613 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-5ee26154-aa94-4743-b486-11dc225d86d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:27.785295 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1297/2582] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:43:27 2022] DELETE /identity/v3/projects/ae8deb8074014c20ae8e68fd045830b1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:27.790344 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9d3b1-768b-47ee-99b9-42e54fc4b291 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:27.790810 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9d3b1-768b-47ee-99b9-42e54fc4b291 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:27.790905 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9d3b1-768b-47ee-99b9-42e54fc4b291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:27.809793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7de9d3b1-768b-47ee-99b9-42e54fc4b291 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:27.811043 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7de9d3b1-768b-47ee-99b9-42e54fc4b291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:27.868991 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1286/2583] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:27.876701 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516ab2c-a719-4db9-97b7-6533edb36b5c None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:27.876996 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516ab2c-a719-4db9-97b7-6533edb36b5c None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:27.877281 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516ab2c-a719-4db9-97b7-6533edb36b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:27.895674 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-a516ab2c-a719-4db9-97b7-6533edb36b5c None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:27.896884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-a516ab2c-a719-4db9-97b7-6533edb36b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:27.905414 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1298/2584] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:27.909467 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de9d3b1-768b-47ee-99b9-42e54fc4b291 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:27.919356 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de9d3b1-768b-47ee-99b9-42e54fc4b291 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:27.923930 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed090d3-98c9-4b48-bf1d-b21caa4d24c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:27.924587 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed090d3-98c9-4b48-bf1d-b21caa4d24c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:27.924882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed090d3-98c9-4b48-bf1d-b21caa4d24c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:27.925170 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed090d3-98c9-4b48-bf1d-b21caa4d24c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:27.925898 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed090d3-98c9-4b48-bf1d-b21caa4d24c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:43:27.926214 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed090d3-98c9-4b48-bf1d-b21caa4d24c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:27.927840 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed090d3-98c9-4b48-bf1d-b21caa4d24c7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:27.939514 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1287/2585] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:43:27.944306 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516ab2c-a719-4db9-97b7-6533edb36b5c None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:27.955209 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516ab2c-a719-4db9-97b7-6533edb36b5c None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:27.959889 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d76eb1-bc3a-4ad2-a3a0-ce8cfbcbe3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:27.960496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d76eb1-bc3a-4ad2-a3a0-ce8cfbcbe3dc admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:27.960786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d76eb1-bc3a-4ad2-a3a0-ce8cfbcbe3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:27.961071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d76eb1-bc3a-4ad2-a3a0-ce8cfbcbe3dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:27.961772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d76eb1-bc3a-4ad2-a3a0-ce8cfbcbe3dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:27.963836 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d76eb1-bc3a-4ad2-a3a0-ce8cfbcbe3dc admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:27.991896 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1299/2586] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:43:27 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:27.993021 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed090d3-98c9-4b48-bf1d-b21caa4d24c7 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:28.003147 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed090d3-98c9-4b48-bf1d-b21caa4d24c7 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:28.031022 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:28.063874 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e37fe4-a75b-42f8-9cbd-0ce22745151e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:28.064548 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e37fe4-a75b-42f8-9cbd-0ce22745151e admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:28.064843 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e37fe4-a75b-42f8-9cbd-0ce22745151e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:28.065126 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e37fe4-a75b-42f8-9cbd-0ce22745151e admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:28.065811 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e37fe4-a75b-42f8-9cbd-0ce22745151e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:28.067785 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e37fe4-a75b-42f8-9cbd-0ce22745151e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:28.087831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1288/2587] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:43:27 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:43:28.092917 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d76eb1-bc3a-4ad2-a3a0-ce8cfbcbe3dc admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:28.103207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d76eb1-bc3a-4ad2-a3a0-ce8cfbcbe3dc admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:28.108287 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa897e-aec8-46cc-a7d4-e6f867d506a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:28.108951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa897e-aec8-46cc-a7d4-e6f867d506a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:28.109234 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa897e-aec8-46cc-a7d4-e6f867d506a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:28.109515 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa897e-aec8-46cc-a7d4-e6f867d506a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:28.110271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa897e-aec8-46cc-a7d4-e6f867d506a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:28.111900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa897e-aec8-46cc-a7d4-e6f867d506a1 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:28.124057 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1300/2588] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:28.128700 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e37fe4-a75b-42f8-9cbd-0ce22745151e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:28.139681 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e37fe4-a75b-42f8-9cbd-0ce22745151e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:28.144354 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536b7c7-c098-4304-85ab-661f8d98a129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:28.145040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536b7c7-c098-4304-85ab-661f8d98a129 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:28.145327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536b7c7-c098-4304-85ab-661f8d98a129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:28.145608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536b7c7-c098-4304-85ab-661f8d98a129 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:28.146332 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536b7c7-c098-4304-85ab-661f8d98a129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:28.147940 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536b7c7-c098-4304-85ab-661f8d98a129 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:28.157970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1289/2589] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:28.162060 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa897e-aec8-46cc-a7d4-e6f867d506a1 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:28.172738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa897e-aec8-46cc-a7d4-e6f867d506a1 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:28.177296 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f017b47-8f53-4b36-85d9-c6f202ae13ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:28.178286 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f017b47-8f53-4b36-85d9-c6f202ae13ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:28.178578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f017b47-8f53-4b36-85d9-c6f202ae13ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:28.178855 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f017b47-8f53-4b36-85d9-c6f202ae13ca admin admin] PATH_INFO: `/v3/projects/5f871727976b4778b8aaa3115c03be4f/users/f4b3019facf345258b0e5b3f22cca62a/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:28.194638 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f017b47-8f53-4b36-85d9-c6f202ae13ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f871727976b4778b8aaa3115c03be4f, user_id=f4b3019facf345258b0e5b3f22cca62a, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:28.197290 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f017b47-8f53-4b36-85d9-c6f202ae13ca admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:28.212795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1301/2590] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:28 2022] PUT /identity/v3/projects/5f871727976b4778b8aaa3115c03be4f/users/f4b3019facf345258b0e5b3f22cca62a/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:43:28.223068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1e7a5-fc72-4bb1-8c05-03028bb64d32 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:28.223362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1e7a5-fc72-4bb1-8c05-03028bb64d32 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:28.223669 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1e7a5-fc72-4bb1-8c05-03028bb64d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:28.249689 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-27e1e7a5-fc72-4bb1-8c05-03028bb64d32 None None] MFA Rules not processed for user `f4b3019facf345258b0e5b3f22cca62a`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:28.250897 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-27e1e7a5-fc72-4bb1-8c05-03028bb64d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:28.313176 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1290/2591] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:28.317574 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f017b47-8f53-4b36-85d9-c6f202ae13ca admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:28.378215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f017b47-8f53-4b36-85d9-c6f202ae13ca admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:28.382875 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f7466-08c5-4176-962b-f97bcfa661cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:28.383564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f7466-08c5-4176-962b-f97bcfa661cd admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:28.383853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f7466-08c5-4176-962b-f97bcfa661cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:28.384133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f7466-08c5-4176-962b-f97bcfa661cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:28.384863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f7466-08c5-4176-962b-f97bcfa661cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:28.387551 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f7466-08c5-4176-962b-f97bcfa661cd admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:28.413465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1302/2592] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:43:28 2022] POST /identity/v3/projects => generated 363 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:28.417943 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1e7a5-fc72-4bb1-8c05-03028bb64d32 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:28.428659 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1e7a5-fc72-4bb1-8c05-03028bb64d32 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:28.487862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e68d8-68d8-4e66-939f-a732c71a7d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:28.488538 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e68d8-68d8-4e66-939f-a732c71a7d3e admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:28.488854 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e68d8-68d8-4e66-939f-a732c71a7d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:28.489139 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e68d8-68d8-4e66-939f-a732c71a7d3e admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:28.489822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e68d8-68d8-4e66-939f-a732c71a7d3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:28.491795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e68d8-68d8-4e66-939f-a732c71a7d3e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:28.514092 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1291/2593] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:43:28 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:43:28.518215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f7466-08c5-4176-962b-f97bcfa661cd admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:28.528951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f7466-08c5-4176-962b-f97bcfa661cd admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:28.533829 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac7162-b39a-4470-9979-aa80b0700be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:28.534492 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac7162-b39a-4470-9979-aa80b0700be7 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:28.534777 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac7162-b39a-4470-9979-aa80b0700be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:28.535134 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac7162-b39a-4470-9979-aa80b0700be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:28.535887 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ac7162-b39a-4470-9979-aa80b0700be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:28.537466 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ac7162-b39a-4470-9979-aa80b0700be7 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:28.549015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1303/2594] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:28.553179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e68d8-68d8-4e66-939f-a732c71a7d3e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:28.563935 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e68d8-68d8-4e66-939f-a732c71a7d3e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:28.568887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcf966-0551-45ad-b096-efb1bbebf8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:28.569764 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcf966-0551-45ad-b096-efb1bbebf8c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:28.570052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcf966-0551-45ad-b096-efb1bbebf8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:28.570454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcf966-0551-45ad-b096-efb1bbebf8c9 admin admin] PATH_INFO: `/v3/projects/42cdc768d2534b33b34e10d574468d6e/users/7f3a581b1156438089870c1da88a26f4/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:28.585430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcf966-0551-45ad-b096-efb1bbebf8c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42cdc768d2534b33b34e10d574468d6e, user_id=7f3a581b1156438089870c1da88a26f4, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:28.588096 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcf966-0551-45ad-b096-efb1bbebf8c9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:28.601190 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1292/2595] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:28 2022] PUT /identity/v3/projects/42cdc768d2534b33b34e10d574468d6e/users/7f3a581b1156438089870c1da88a26f4/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:43:28.605669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac7162-b39a-4470-9979-aa80b0700be7 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:28.616830 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac7162-b39a-4470-9979-aa80b0700be7 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:28.637410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:28.675319 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440f414-446d-4769-a64a-0cbfa4d9da2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:28.675992 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440f414-446d-4769-a64a-0cbfa4d9da2d admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:28.676285 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440f414-446d-4769-a64a-0cbfa4d9da2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:28.676570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440f414-446d-4769-a64a-0cbfa4d9da2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:28.677295 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440f414-446d-4769-a64a-0cbfa4d9da2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:28.678885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7440f414-446d-4769-a64a-0cbfa4d9da2d admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:28.689295 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1304/2596] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:28.693544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcf966-0551-45ad-b096-efb1bbebf8c9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:28.700088 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:28.706384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcf966-0551-45ad-b096-efb1bbebf8c9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:28.712864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43fdf6-14a3-4efe-8a5f-04c51996cc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:28.713937 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43fdf6-14a3-4efe-8a5f-04c51996cc18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:28.714293 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43fdf6-14a3-4efe-8a5f-04c51996cc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:28.714665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43fdf6-14a3-4efe-8a5f-04c51996cc18 admin admin] PATH_INFO: `/v3/projects/42cdc768d2534b33b34e10d574468d6e/users/7f3a581b1156438089870c1da88a26f4/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:28.716730 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43fdf6-14a3-4efe-8a5f-04c51996cc18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42cdc768d2534b33b34e10d574468d6e, user_id=7f3a581b1156438089870c1da88a26f4, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:28.719478 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43fdf6-14a3-4efe-8a5f-04c51996cc18 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:28.730045 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1293/2597] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:28 2022] PUT /identity/v3/projects/42cdc768d2534b33b34e10d574468d6e/users/7f3a581b1156438089870c1da88a26f4/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:28.742825 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b875b5-4515-45a2-8c64-16655fe04c75 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:28.743086 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b875b5-4515-45a2-8c64-16655fe04c75 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:28.743377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b875b5-4515-45a2-8c64-16655fe04c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:28.772800 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-19b875b5-4515-45a2-8c64-16655fe04c75 None None] MFA Rules not processed for user `7f3a581b1156438089870c1da88a26f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:28.774035 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-19b875b5-4515-45a2-8c64-16655fe04c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:28.848580 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1305/2598] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:28 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:28.860596 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43fdf6-14a3-4efe-8a5f-04c51996cc18 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:28.862478 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ed43fdf6-14a3-4efe-8a5f-04c51996cc18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:28.920155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43fdf6-14a3-4efe-8a5f-04c51996cc18 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:28.923839 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d56412-c2af-4333-9326-3cf8bd323b85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:28.924495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d56412-c2af-4333-9326-3cf8bd323b85 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:28.925397 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d56412-c2af-4333-9326-3cf8bd323b85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:28.925694 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d56412-c2af-4333-9326-3cf8bd323b85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:28.935022 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d56412-c2af-4333-9326-3cf8bd323b85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:28.936447 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d56412-c2af-4333-9326-3cf8bd323b85 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:28.946719 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1294/2599] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:28 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:29.020184 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b875b5-4515-45a2-8c64-16655fe04c75 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:29.030859 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b875b5-4515-45a2-8c64-16655fe04c75 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:29.034808 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867e5a7-accd-4a55-8e1d-26bc0f0e59bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:29.035500 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867e5a7-accd-4a55-8e1d-26bc0f0e59bb service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:29.035822 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867e5a7-accd-4a55-8e1d-26bc0f0e59bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:29.036111 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867e5a7-accd-4a55-8e1d-26bc0f0e59bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:29.094329 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b867e5a7-accd-4a55-8e1d-26bc0f0e59bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:29.095814 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b867e5a7-accd-4a55-8e1d-26bc0f0e59bb service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:29.104624 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1306/2600] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:37.574002 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e551a24-01b3-411f-89b0-f3c452b9cb0c None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:37.574493 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e551a24-01b3-411f-89b0-f3c452b9cb0c None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:37.574493 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e551a24-01b3-411f-89b0-f3c452b9cb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:37.625185 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-9e551a24-01b3-411f-89b0-f3c452b9cb0c None None] MFA Rules not processed for user `df1aeb2be41c49819b811ecfa47d4155`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:37.626594 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-9e551a24-01b3-411f-89b0-f3c452b9cb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:37.691519 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1295/2601] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 22:43:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:43:37.740169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867e5a7-accd-4a55-8e1d-26bc0f0e59bb service cinder] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:37.741589 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b867e5a7-accd-4a55-8e1d-26bc0f0e59bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:37.794294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:37.841810 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867e5a7-accd-4a55-8e1d-26bc0f0e59bb service cinder] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:37.846874 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e44821-f000-42f6-bcb3-c3e2718d89fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ce35c43e8294f4cbc880f2219a6eb76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:37.847578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e44821-f000-42f6-bcb3-c3e2718d89fe service swift] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:37.847864 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e44821-f000-42f6-bcb3-c3e2718d89fe service swift] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:37.848148 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e44821-f000-42f6-bcb3-c3e2718d89fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:37.856687 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e44821-f000-42f6-bcb3-c3e2718d89fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:37.858364 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e44821-f000-42f6-bcb3-c3e2718d89fe service swift] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:37.866582 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1307/2602] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 22:43:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:43:38.285943 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e551a24-01b3-411f-89b0-f3c452b9cb0c None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:38.379206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e551a24-01b3-411f-89b0-f3c452b9cb0c None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:38.385364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fb0fe-b4d8-422f-9827-0d208ecad7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:38.386645 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fb0fe-b4d8-422f-9827-0d208ecad7c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:38.387049 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fb0fe-b4d8-422f-9827-0d208ecad7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:38.387470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fb0fe-b4d8-422f-9827-0d208ecad7c5 admin admin] PATH_INFO: `/v3/users/f4b3019facf345258b0e5b3f22cca62a` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:38.389170 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fb0fe-b4d8-422f-9827-0d208ecad7c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4b3019facf345258b0e5b3f22cca62a)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:38.391718 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fb0fe-b4d8-422f-9827-0d208ecad7c5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:38.398484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:38.460293 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-327fb0fe-b4d8-422f-9827-0d208ecad7c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4b3019facf345258b0e5b3f22cca62a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:38.477863 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-327fb0fe-b4d8-422f-9827-0d208ecad7c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4b3019facf345258b0e5b3f22cca62a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:38.487043 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-327fb0fe-b4d8-422f-9827-0d208ecad7c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4b3019facf345258b0e5b3f22cca62a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:38.487856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-327fb0fe-b4d8-422f-9827-0d208ecad7c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4b3019facf345258b0e5b3f22cca62a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:38.488746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-327fb0fe-b4d8-422f-9827-0d208ecad7c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4b3019facf345258b0e5b3f22cca62a'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:38.513365 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1296/2603] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:43:38 2022] DELETE /identity/v3/users/f4b3019facf345258b0e5b3f22cca62a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:38.517995 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad8d58-4f96-49ba-bb30-5a2de9cad322 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:38.518299 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad8d58-4f96-49ba-bb30-5a2de9cad322 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:38.518592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad8d58-4f96-49ba-bb30-5a2de9cad322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:38.537832 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-bbad8d58-4f96-49ba-bb30-5a2de9cad322 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:38.539093 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-bbad8d58-4f96-49ba-bb30-5a2de9cad322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:38.616497 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1308/2604] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:38.623274 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fb0fe-b4d8-422f-9827-0d208ecad7c5 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:38.624746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-327fb0fe-b4d8-422f-9827-0d208ecad7c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:38.692336 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fb0fe-b4d8-422f-9827-0d208ecad7c5 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:38.697894 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335f8ec-0830-41bb-9fe4-2f8f1bba5539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:38.698699 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335f8ec-0830-41bb-9fe4-2f8f1bba5539 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:38.699101 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335f8ec-0830-41bb-9fe4-2f8f1bba5539 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:38.699472 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335f8ec-0830-41bb-9fe4-2f8f1bba5539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:38.710552 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335f8ec-0830-41bb-9fe4-2f8f1bba5539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:38.712138 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335f8ec-0830-41bb-9fe4-2f8f1bba5539 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:38.724119 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1297/2605] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:39.063630 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad8d58-4f96-49ba-bb30-5a2de9cad322 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:39.065556 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-bbad8d58-4f96-49ba-bb30-5a2de9cad322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:39.112144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad8d58-4f96-49ba-bb30-5a2de9cad322 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:39.117804 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a2440-1a75-4b27-a796-d390ede47e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:39.119390 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a2440-1a75-4b27-a796-d390ede47e14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:39.119893 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a2440-1a75-4b27-a796-d390ede47e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:39.120321 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a2440-1a75-4b27-a796-d390ede47e14 admin admin] PATH_INFO: `/v3/projects/5f871727976b4778b8aaa3115c03be4f` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:39.125537 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a2440-1a75-4b27-a796-d390ede47e14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f871727976b4778b8aaa3115c03be4f)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:39.125537 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a2440-1a75-4b27-a796-d390ede47e14 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:39.190994 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-dd2a2440-1a75-4b27-a796-d390ede47e14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f871727976b4778b8aaa3115c03be4f', 'request_id': 'req-dd2a2440-1a75-4b27-a796-d390ede47e14'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:39.204451 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-dd2a2440-1a75-4b27-a796-d390ede47e14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f871727976b4778b8aaa3115c03be4f', 'request_id': 'req-dd2a2440-1a75-4b27-a796-d390ede47e14'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:39.216564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-dd2a2440-1a75-4b27-a796-d390ede47e14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f871727976b4778b8aaa3115c03be4f', 'request_id': 'req-dd2a2440-1a75-4b27-a796-d390ede47e14'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:39.228452 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-dd2a2440-1a75-4b27-a796-d390ede47e14 admin admin] The token cache is being invalidate because project 5f871727976b4778b8aaa3115c03be4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:43:39.228747 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-dd2a2440-1a75-4b27-a796-d390ede47e14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:39.229477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-dd2a2440-1a75-4b27-a796-d390ede47e14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:39.233637 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1309/2606] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:43:39 2022] DELETE /identity/v3/projects/5f871727976b4778b8aaa3115c03be4f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:39.237781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335f8ec-0830-41bb-9fe4-2f8f1bba5539 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:39.248648 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335f8ec-0830-41bb-9fe4-2f8f1bba5539 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:39.318729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a78707-7822-4eba-871c-fd9ba531e171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:39.319654 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a78707-7822-4eba-871c-fd9ba531e171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:39.319947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a78707-7822-4eba-871c-fd9ba531e171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:39.320239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a78707-7822-4eba-871c-fd9ba531e171 admin admin] PATH_INFO: `/v3/users/7f3a581b1156438089870c1da88a26f4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:39.321614 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a78707-7822-4eba-871c-fd9ba531e171 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f3a581b1156438089870c1da88a26f4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:39.323396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a78707-7822-4eba-871c-fd9ba531e171 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:39.374910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:39.398003 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a9a78707-7822-4eba-871c-fd9ba531e171 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f3a581b1156438089870c1da88a26f4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:39.406728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a9a78707-7822-4eba-871c-fd9ba531e171 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f3a581b1156438089870c1da88a26f4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:39.420216 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a9a78707-7822-4eba-871c-fd9ba531e171 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f3a581b1156438089870c1da88a26f4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:39.420919 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a9a78707-7822-4eba-871c-fd9ba531e171 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f3a581b1156438089870c1da88a26f4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:39.421731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-a9a78707-7822-4eba-871c-fd9ba531e171 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f3a581b1156438089870c1da88a26f4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:39.456648 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1298/2607] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:43:39 2022] DELETE /identity/v3/users/7f3a581b1156438089870c1da88a26f4 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:39.795475 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a2440-1a75-4b27-a796-d390ede47e14 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:39.797239 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-dd2a2440-1a75-4b27-a796-d390ede47e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:39.818505 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a2440-1a75-4b27-a796-d390ede47e14 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:39.865006 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:39.895380 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e8005-dcef-4d63-876a-c00b1cb3b6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:39.897287 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e8005-dcef-4d63-876a-c00b1cb3b6c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:39.897583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e8005-dcef-4d63-876a-c00b1cb3b6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:39.898044 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e8005-dcef-4d63-876a-c00b1cb3b6c9 admin admin] PATH_INFO: `/v3/projects/42cdc768d2534b33b34e10d574468d6e` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:39.899360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e8005-dcef-4d63-876a-c00b1cb3b6c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42cdc768d2534b33b34e10d574468d6e)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:39.901504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e8005-dcef-4d63-876a-c00b1cb3b6c9 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:39.961985 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-278e8005-dcef-4d63-876a-c00b1cb3b6c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42cdc768d2534b33b34e10d574468d6e', 'request_id': 'req-278e8005-dcef-4d63-876a-c00b1cb3b6c9'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:39.971642 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-278e8005-dcef-4d63-876a-c00b1cb3b6c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42cdc768d2534b33b34e10d574468d6e', 'request_id': 'req-278e8005-dcef-4d63-876a-c00b1cb3b6c9'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:39.983108 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-278e8005-dcef-4d63-876a-c00b1cb3b6c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42cdc768d2534b33b34e10d574468d6e', 'request_id': 'req-278e8005-dcef-4d63-876a-c00b1cb3b6c9'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:39.997329 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-278e8005-dcef-4d63-876a-c00b1cb3b6c9 admin admin] The token cache is being invalidate because project 42cdc768d2534b33b34e10d574468d6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:43:39.997633 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-278e8005-dcef-4d63-876a-c00b1cb3b6c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:39.998287 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-278e8005-dcef-4d63-876a-c00b1cb3b6c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:40.000423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1310/2608] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:43:39 2022] DELETE /identity/v3/projects/42cdc768d2534b33b34e10d574468d6e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:43:40.005084 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98054cbf-0fa9-43dc-9668-0eb2f28cbeed None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:40.007655 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98054cbf-0fa9-43dc-9668-0eb2f28cbeed None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:40.007994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98054cbf-0fa9-43dc-9668-0eb2f28cbeed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:40.030407 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-98054cbf-0fa9-43dc-9668-0eb2f28cbeed None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:40.031764 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-98054cbf-0fa9-43dc-9668-0eb2f28cbeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:40.111013 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1299/2609] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:43:40.113272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcf899-6aea-4024-966b-3f047c12eb0d None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:40.113732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcf899-6aea-4024-966b-3f047c12eb0d None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:40.113852 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcf899-6aea-4024-966b-3f047c12eb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:40.134672 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-98fcf899-6aea-4024-966b-3f047c12eb0d None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:40.136328 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-98fcf899-6aea-4024-966b-3f047c12eb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:40.146572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1311/2610] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:40.150738 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98054cbf-0fa9-43dc-9668-0eb2f28cbeed None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:40.167097 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98054cbf-0fa9-43dc-9668-0eb2f28cbeed None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:40.167540 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457b12a-086c-4934-971a-3040b84fd5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:40.170798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457b12a-086c-4934-971a-3040b84fd5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:40.170798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457b12a-086c-4934-971a-3040b84fd5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:40.170798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457b12a-086c-4934-971a-3040b84fd5f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:40.170798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457b12a-086c-4934-971a-3040b84fd5f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:43:40.170798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457b12a-086c-4934-971a-3040b84fd5f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:40.172246 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457b12a-086c-4934-971a-3040b84fd5f3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:40.188748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1300/2611] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:43:40.193166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcf899-6aea-4024-966b-3f047c12eb0d None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:40.205030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcf899-6aea-4024-966b-3f047c12eb0d None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:40.208487 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b1e2b-f4c9-4799-8000-022546e049ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:40.209270 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b1e2b-f4c9-4799-8000-022546e049ef admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:40.209728 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b1e2b-f4c9-4799-8000-022546e049ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:40.210079 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b1e2b-f4c9-4799-8000-022546e049ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:40.210846 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01b1e2b-f4c9-4799-8000-022546e049ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:40.212862 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01b1e2b-f4c9-4799-8000-022546e049ef admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:40.247955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1312/2612] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:43:40 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:43:40.248155 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457b12a-086c-4934-971a-3040b84fd5f3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:40.258792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457b12a-086c-4934-971a-3040b84fd5f3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:40.336801 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764a6d3-ff7b-46d4-927f-5047cabaef54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:40.337527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764a6d3-ff7b-46d4-927f-5047cabaef54 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:40.337815 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764a6d3-ff7b-46d4-927f-5047cabaef54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:40.338100 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764a6d3-ff7b-46d4-927f-5047cabaef54 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:40.338798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764a6d3-ff7b-46d4-927f-5047cabaef54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:40.341293 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764a6d3-ff7b-46d4-927f-5047cabaef54 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:40.371221 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1301/2613] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:43:40 2022] POST /identity/v3/users => generated 290 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:40.375495 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b1e2b-f4c9-4799-8000-022546e049ef admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:40.387197 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b1e2b-f4c9-4799-8000-022546e049ef admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:40.393513 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdd0cb-82e0-47d7-8c54-7baad2670fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:40.394270 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdd0cb-82e0-47d7-8c54-7baad2670fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:40.394630 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdd0cb-82e0-47d7-8c54-7baad2670fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:40.395020 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdd0cb-82e0-47d7-8c54-7baad2670fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:40.395833 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdd0cb-82e0-47d7-8c54-7baad2670fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:40.397638 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdd0cb-82e0-47d7-8c54-7baad2670fc0 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:40.412537 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1313/2614] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:40.416821 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764a6d3-ff7b-46d4-927f-5047cabaef54 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:40.430818 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764a6d3-ff7b-46d4-927f-5047cabaef54 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:40.437692 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10161d29-d8f0-49f7-bb1f-48997e579925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:40.439052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10161d29-d8f0-49f7-bb1f-48997e579925 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:40.439342 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10161d29-d8f0-49f7-bb1f-48997e579925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:40.439626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10161d29-d8f0-49f7-bb1f-48997e579925 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:40.440401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10161d29-d8f0-49f7-bb1f-48997e579925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:40.441994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10161d29-d8f0-49f7-bb1f-48997e579925 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:40.454130 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1302/2615] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:43:40.458123 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdd0cb-82e0-47d7-8c54-7baad2670fc0 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:40.469274 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdd0cb-82e0-47d7-8c54-7baad2670fc0 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:40.476991 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d460b-1f1e-41fb-936e-7cfa68cbe929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:40.476991 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d460b-1f1e-41fb-936e-7cfa68cbe929 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:40.476991 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d460b-1f1e-41fb-936e-7cfa68cbe929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:40.476991 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d460b-1f1e-41fb-936e-7cfa68cbe929 admin admin] PATH_INFO: `/v3/projects/f3d44990af8246a0b3b3af11b2a54809/users/310483352f374f49a816ceeec0363a57/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:40.492314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d460b-1f1e-41fb-936e-7cfa68cbe929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3d44990af8246a0b3b3af11b2a54809, user_id=310483352f374f49a816ceeec0363a57, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:40.496642 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d460b-1f1e-41fb-936e-7cfa68cbe929 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:40.514437 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1314/2616] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:40 2022] PUT /identity/v3/projects/f3d44990af8246a0b3b3af11b2a54809/users/310483352f374f49a816ceeec0363a57/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:40.526550 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9686e-6610-4377-aa99-a19e9515aa01 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:40.526838 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9686e-6610-4377-aa99-a19e9515aa01 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:40.527167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9686e-6610-4377-aa99-a19e9515aa01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:40.568013 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:40.580311 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-22c9686e-6610-4377-aa99-a19e9515aa01 None None] MFA Rules not processed for user `310483352f374f49a816ceeec0363a57`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:40.582194 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-22c9686e-6610-4377-aa99-a19e9515aa01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:40.668221 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1303/2617] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:40.673042 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d460b-1f1e-41fb-936e-7cfa68cbe929 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:40.743537 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d460b-1f1e-41fb-936e-7cfa68cbe929 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:40.748445 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057df55-02c5-4ca7-b6f8-dbd3cdcee6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:40.749145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057df55-02c5-4ca7-b6f8-dbd3cdcee6c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:40.749439 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057df55-02c5-4ca7-b6f8-dbd3cdcee6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:40.749723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057df55-02c5-4ca7-b6f8-dbd3cdcee6c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:40.750440 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057df55-02c5-4ca7-b6f8-dbd3cdcee6c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:40.752377 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057df55-02c5-4ca7-b6f8-dbd3cdcee6c1 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:40.761726 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:40.787083 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1315/2618] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:43:40 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:43:40.789639 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9686e-6610-4377-aa99-a19e9515aa01 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:40.802303 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9686e-6610-4377-aa99-a19e9515aa01 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:40.893266 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40bdbca-be41-45f5-beeb-1f640981275e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:40.893959 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bdbca-be41-45f5-beeb-1f640981275e admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:40.894251 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bdbca-be41-45f5-beeb-1f640981275e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:40.894540 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40bdbca-be41-45f5-beeb-1f640981275e admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:40.895282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40bdbca-be41-45f5-beeb-1f640981275e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:40.897208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40bdbca-be41-45f5-beeb-1f640981275e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:40.922171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1304/2619] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:43:40 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:40.927697 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057df55-02c5-4ca7-b6f8-dbd3cdcee6c1 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:40.940477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057df55-02c5-4ca7-b6f8-dbd3cdcee6c1 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:40.948397 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d6b9d-bc10-442e-a217-411bff711efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:40.949443 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d6b9d-bc10-442e-a217-411bff711efa admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:40.949733 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d6b9d-bc10-442e-a217-411bff711efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:40.950087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d6b9d-bc10-442e-a217-411bff711efa admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:40.952312 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d6b9d-bc10-442e-a217-411bff711efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:40.953951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d6b9d-bc10-442e-a217-411bff711efa admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:40.967782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1316/2620] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:40.972613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40bdbca-be41-45f5-beeb-1f640981275e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:40.983382 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40bdbca-be41-45f5-beeb-1f640981275e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:40.988278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3425de-f3a4-4f2b-a34e-9198f0914b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:40.988988 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3425de-f3a4-4f2b-a34e-9198f0914b79 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:40.989284 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3425de-f3a4-4f2b-a34e-9198f0914b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:40.989576 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3425de-f3a4-4f2b-a34e-9198f0914b79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:40.990351 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3425de-f3a4-4f2b-a34e-9198f0914b79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:40.992055 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3425de-f3a4-4f2b-a34e-9198f0914b79 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:41.006644 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1305/2621] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:41.008294 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d6b9d-bc10-442e-a217-411bff711efa admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:41.021087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d6b9d-bc10-442e-a217-411bff711efa admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:41.031208 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c1eae-5344-4aca-80c4-68026cbf137e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:41.032160 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c1eae-5344-4aca-80c4-68026cbf137e admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:41.032588 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c1eae-5344-4aca-80c4-68026cbf137e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:41.032882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c1eae-5344-4aca-80c4-68026cbf137e admin admin] PATH_INFO: `/v3/projects/0ea7368e39f343b68eb92c3b6a5f730a/users/ef91b042f9894f84a495db891af96e9e/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:41.054056 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c1eae-5344-4aca-80c4-68026cbf137e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea7368e39f343b68eb92c3b6a5f730a, user_id=ef91b042f9894f84a495db891af96e9e, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:41.056760 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c1eae-5344-4aca-80c4-68026cbf137e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:41.073512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1317/2622] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:41 2022] PUT /identity/v3/projects/0ea7368e39f343b68eb92c3b6a5f730a/users/ef91b042f9894f84a495db891af96e9e/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:43:41.086260 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61f49f-b513-4be7-9bc5-f055346c6078 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:41.086804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61f49f-b513-4be7-9bc5-f055346c6078 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:41.087434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61f49f-b513-4be7-9bc5-f055346c6078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:41.116636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8e61f49f-b513-4be7-9bc5-f055346c6078 None None] MFA Rules not processed for user `ef91b042f9894f84a495db891af96e9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:41.118204 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8e61f49f-b513-4be7-9bc5-f055346c6078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:41.203633 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1306/2623] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:41.208525 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c1eae-5344-4aca-80c4-68026cbf137e admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:41.284911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c1eae-5344-4aca-80c4-68026cbf137e admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:41.289879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231aa6d9-ecd3-4e5b-8ea1-f50d9878a5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:41.290570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231aa6d9-ecd3-4e5b-8ea1-f50d9878a5f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:41.290858 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231aa6d9-ecd3-4e5b-8ea1-f50d9878a5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:41.291170 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231aa6d9-ecd3-4e5b-8ea1-f50d9878a5f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:41.292161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231aa6d9-ecd3-4e5b-8ea1-f50d9878a5f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:41.294834 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231aa6d9-ecd3-4e5b-8ea1-f50d9878a5f9 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:41.324187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1318/2624] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:43:41 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:41.329576 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61f49f-b513-4be7-9bc5-f055346c6078 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:41.341556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61f49f-b513-4be7-9bc5-f055346c6078 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:41.346684 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:41.409998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8eb42-d99a-4c5a-bec8-4c5ccdeb184d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:41.410674 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8eb42-d99a-4c5a-bec8-4c5ccdeb184d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:41.410994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8eb42-d99a-4c5a-bec8-4c5ccdeb184d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:41.411292 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8eb42-d99a-4c5a-bec8-4c5ccdeb184d admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:41.411970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8eb42-d99a-4c5a-bec8-4c5ccdeb184d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:41.413886 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8eb42-d99a-4c5a-bec8-4c5ccdeb184d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:41.437459 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1307/2625] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:43:41 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:41.442129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231aa6d9-ecd3-4e5b-8ea1-f50d9878a5f9 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:41.454125 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231aa6d9-ecd3-4e5b-8ea1-f50d9878a5f9 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:41.461108 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d5596-06fc-441f-97d0-8815f7740135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:41.461900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d5596-06fc-441f-97d0-8815f7740135 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:41.462190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d5596-06fc-441f-97d0-8815f7740135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:41.462483 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d5596-06fc-441f-97d0-8815f7740135 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:41.463256 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d5596-06fc-441f-97d0-8815f7740135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:41.465188 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d5596-06fc-441f-97d0-8815f7740135 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:41.481071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1319/2626] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:43:41.482447 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8eb42-d99a-4c5a-bec8-4c5ccdeb184d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:41.494362 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8eb42-d99a-4c5a-bec8-4c5ccdeb184d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:41.500120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e49e0-6674-48cf-a2e1-dd34aae53680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:41.501030 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e49e0-6674-48cf-a2e1-dd34aae53680 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:41.501319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e49e0-6674-48cf-a2e1-dd34aae53680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:41.501599 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e49e0-6674-48cf-a2e1-dd34aae53680 admin admin] PATH_INFO: `/v3/projects/ce6d99420d6c4b349945e6ead1af603a/users/436f85bfb55a4d1a87b38a9ccc4e8903/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:41.522524 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e49e0-6674-48cf-a2e1-dd34aae53680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce6d99420d6c4b349945e6ead1af603a, user_id=436f85bfb55a4d1a87b38a9ccc4e8903, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:41.527569 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e49e0-6674-48cf-a2e1-dd34aae53680 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:41.547064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1308/2627] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:41 2022] PUT /identity/v3/projects/ce6d99420d6c4b349945e6ead1af603a/users/436f85bfb55a4d1a87b38a9ccc4e8903/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:41.548632 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d5596-06fc-441f-97d0-8815f7740135 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:41.563130 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d5596-06fc-441f-97d0-8815f7740135 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:41.643057 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed4ed3-2725-4e7f-8ef4-0e1d4a05a644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:41.644089 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed4ed3-2725-4e7f-8ef4-0e1d4a05a644 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:41.644535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed4ed3-2725-4e7f-8ef4-0e1d4a05a644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:41.645041 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed4ed3-2725-4e7f-8ef4-0e1d4a05a644 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:41.645928 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed4ed3-2725-4e7f-8ef4-0e1d4a05a644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:41.648614 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed4ed3-2725-4e7f-8ef4-0e1d4a05a644 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:41.660651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1320/2628] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:41.665732 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e49e0-6674-48cf-a2e1-dd34aae53680 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:41.678845 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e49e0-6674-48cf-a2e1-dd34aae53680 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:41.684068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e9439-d2ba-4616-aa63-e8532fc06cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:41.690717 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e9439-d2ba-4616-aa63-e8532fc06cee admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:41.691125 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e9439-d2ba-4616-aa63-e8532fc06cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:41.691615 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e9439-d2ba-4616-aa63-e8532fc06cee admin admin] PATH_INFO: `/v3/projects/ce6d99420d6c4b349945e6ead1af603a/users/436f85bfb55a4d1a87b38a9ccc4e8903/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:41.694026 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e9439-d2ba-4616-aa63-e8532fc06cee admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce6d99420d6c4b349945e6ead1af603a, user_id=436f85bfb55a4d1a87b38a9ccc4e8903, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:41.697009 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e9439-d2ba-4616-aa63-e8532fc06cee admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:41.714075 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1309/2629] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:41 2022] PUT /identity/v3/projects/ce6d99420d6c4b349945e6ead1af603a/users/436f85bfb55a4d1a87b38a9ccc4e8903/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:41.724168 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d8a5d-688c-4b15-ae1e-b7243af55ea4 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:41.724480 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d8a5d-688c-4b15-ae1e-b7243af55ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:41.724747 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d8a5d-688c-4b15-ae1e-b7243af55ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:41.758772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-bc1d8a5d-688c-4b15-ae1e-b7243af55ea4 None None] MFA Rules not processed for user `436f85bfb55a4d1a87b38a9ccc4e8903`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:41.760322 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-bc1d8a5d-688c-4b15-ae1e-b7243af55ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:41.801997 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:41.936213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1321/2630] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:43:41.956127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e9439-d2ba-4616-aa63-e8532fc06cee admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:41.958149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d03e9439-d2ba-4616-aa63-e8532fc06cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:42.035625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e9439-d2ba-4616-aa63-e8532fc06cee admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:42.039378 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc07caf-3bf8-4d7f-8f4a-f7cb7b374e8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:42.040054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc07caf-3bf8-4d7f-8f4a-f7cb7b374e8b service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:42.040343 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc07caf-3bf8-4d7f-8f4a-f7cb7b374e8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:42.040637 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc07caf-3bf8-4d7f-8f4a-f7cb7b374e8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:42.051325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc07caf-3bf8-4d7f-8f4a-f7cb7b374e8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:42.053254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc07caf-3bf8-4d7f-8f4a-f7cb7b374e8b service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:42.072198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1310/2631] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:42.116653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d8a5d-688c-4b15-ae1e-b7243af55ea4 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:42.118350 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-bc1d8a5d-688c-4b15-ae1e-b7243af55ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:42.131835 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d8a5d-688c-4b15-ae1e-b7243af55ea4 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:42.150647 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e38ac5-13c8-49e5-a50b-b3cc00fa419b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:42.150647 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e38ac5-13c8-49e5-a50b-b3cc00fa419b service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:42.150647 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e38ac5-13c8-49e5-a50b-b3cc00fa419b service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:42.150647 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e38ac5-13c8-49e5-a50b-b3cc00fa419b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:42.235791 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e38ac5-13c8-49e5-a50b-b3cc00fa419b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:42.237372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e38ac5-13c8-49e5-a50b-b3cc00fa419b service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:42.249661 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1322/2632] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:45.270697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc07caf-3bf8-4d7f-8f4a-f7cb7b374e8b service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:45.352268 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc07caf-3bf8-4d7f-8f4a-f7cb7b374e8b service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:45.357511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d36ca-aa93-496a-9813-80becf051450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:45.358489 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d36ca-aa93-496a-9813-80becf051450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:45.358788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d36ca-aa93-496a-9813-80becf051450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:45.359110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d36ca-aa93-496a-9813-80becf051450 admin admin] PATH_INFO: `/v3/users/310483352f374f49a816ceeec0363a57` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:45.360538 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d36ca-aa93-496a-9813-80becf051450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=310483352f374f49a816ceeec0363a57)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:45.362963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d36ca-aa93-496a-9813-80becf051450 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:45.369285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:45.430949 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-041d36ca-aa93-496a-9813-80becf051450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '310483352f374f49a816ceeec0363a57'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:45.447924 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-041d36ca-aa93-496a-9813-80becf051450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '310483352f374f49a816ceeec0363a57'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:45.461014 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-041d36ca-aa93-496a-9813-80becf051450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '310483352f374f49a816ceeec0363a57'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:45.461953 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-041d36ca-aa93-496a-9813-80becf051450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '310483352f374f49a816ceeec0363a57'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:45.462685 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-041d36ca-aa93-496a-9813-80becf051450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '310483352f374f49a816ceeec0363a57'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:45.508822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1311/2633] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:43:45 2022] DELETE /identity/v3/users/310483352f374f49a816ceeec0363a57 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:45.510156 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4f9e1-023f-40e7-9569-26abf98344c4 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:45.510702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4f9e1-023f-40e7-9569-26abf98344c4 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:45.510811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4f9e1-023f-40e7-9569-26abf98344c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:45.531137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-17c4f9e1-023f-40e7-9569-26abf98344c4 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:45.532477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-17c4f9e1-023f-40e7-9569-26abf98344c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:45.623725 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1323/2634] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:43:45.635773 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d36ca-aa93-496a-9813-80becf051450 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:45.637178 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-041d36ca-aa93-496a-9813-80becf051450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:45.715010 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d36ca-aa93-496a-9813-80becf051450 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:45.720154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bad71e-9546-4392-8252-6cd8d79d53ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:45.720939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bad71e-9546-4392-8252-6cd8d79d53ad service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:45.721329 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bad71e-9546-4392-8252-6cd8d79d53ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:45.721777 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bad71e-9546-4392-8252-6cd8d79d53ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:45.735568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bad71e-9546-4392-8252-6cd8d79d53ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:45.737126 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bad71e-9546-4392-8252-6cd8d79d53ad service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:45.750802 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1312/2635] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:46.056886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4f9e1-023f-40e7-9569-26abf98344c4 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:46.059004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-17c4f9e1-023f-40e7-9569-26abf98344c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:46.079785 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4f9e1-023f-40e7-9569-26abf98344c4 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:46.095596 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156afc9-65fa-44e3-8e11-5df2e27996ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:46.096995 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156afc9-65fa-44e3-8e11-5df2e27996ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:46.097382 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156afc9-65fa-44e3-8e11-5df2e27996ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:46.097763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156afc9-65fa-44e3-8e11-5df2e27996ec admin admin] PATH_INFO: `/v3/projects/f3d44990af8246a0b3b3af11b2a54809` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:46.099123 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156afc9-65fa-44e3-8e11-5df2e27996ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3d44990af8246a0b3b3af11b2a54809)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:46.101606 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156afc9-65fa-44e3-8e11-5df2e27996ec admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:46.159688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:46.166372 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d156afc9-65fa-44e3-8e11-5df2e27996ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3d44990af8246a0b3b3af11b2a54809', 'request_id': 'req-d156afc9-65fa-44e3-8e11-5df2e27996ec'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:46.175483 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d156afc9-65fa-44e3-8e11-5df2e27996ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3d44990af8246a0b3b3af11b2a54809', 'request_id': 'req-d156afc9-65fa-44e3-8e11-5df2e27996ec'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:46.185332 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d156afc9-65fa-44e3-8e11-5df2e27996ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3d44990af8246a0b3b3af11b2a54809', 'request_id': 'req-d156afc9-65fa-44e3-8e11-5df2e27996ec'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:46.202215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d156afc9-65fa-44e3-8e11-5df2e27996ec admin admin] The token cache is being invalidate because project f3d44990af8246a0b3b3af11b2a54809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:43:46.202602 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d156afc9-65fa-44e3-8e11-5df2e27996ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:46.203339 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d156afc9-65fa-44e3-8e11-5df2e27996ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:46.204937 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1324/2636] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:43:46 2022] DELETE /identity/v3/projects/f3d44990af8246a0b3b3af11b2a54809 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:46.208845 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bad71e-9546-4392-8252-6cd8d79d53ad service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:46.221261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bad71e-9546-4392-8252-6cd8d79d53ad service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:46.290389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f2f39-7e9b-47cc-81d0-ebaf09707bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:46.291867 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f2f39-7e9b-47cc-81d0-ebaf09707bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:46.292101 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f2f39-7e9b-47cc-81d0-ebaf09707bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:46.292406 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f2f39-7e9b-47cc-81d0-ebaf09707bfc admin admin] PATH_INFO: `/v3/users/ef91b042f9894f84a495db891af96e9e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:46.294860 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f2f39-7e9b-47cc-81d0-ebaf09707bfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef91b042f9894f84a495db891af96e9e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:46.297479 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f2f39-7e9b-47cc-81d0-ebaf09707bfc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:46.334378 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:46.363130 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7c9f2f39-7e9b-47cc-81d0-ebaf09707bfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef91b042f9894f84a495db891af96e9e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:46.370533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7c9f2f39-7e9b-47cc-81d0-ebaf09707bfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef91b042f9894f84a495db891af96e9e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:46.399330 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7c9f2f39-7e9b-47cc-81d0-ebaf09707bfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef91b042f9894f84a495db891af96e9e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:46.400484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7c9f2f39-7e9b-47cc-81d0-ebaf09707bfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef91b042f9894f84a495db891af96e9e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:46.401154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-7c9f2f39-7e9b-47cc-81d0-ebaf09707bfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef91b042f9894f84a495db891af96e9e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:46.432412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1313/2637] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:43:46 2022] DELETE /identity/v3/users/ef91b042f9894f84a495db891af96e9e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:46.745071 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156afc9-65fa-44e3-8e11-5df2e27996ec admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:46.747678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d156afc9-65fa-44e3-8e11-5df2e27996ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:46.768056 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156afc9-65fa-44e3-8e11-5df2e27996ec admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:46.863724 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce4600-8e55-4456-9649-003555e83c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:46.864795 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce4600-8e55-4456-9649-003555e83c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:46.865090 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce4600-8e55-4456-9649-003555e83c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:46.865378 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce4600-8e55-4456-9649-003555e83c1a admin admin] PATH_INFO: `/v3/projects/0ea7368e39f343b68eb92c3b6a5f730a` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:46.866754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce4600-8e55-4456-9649-003555e83c1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ea7368e39f343b68eb92c3b6a5f730a)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:46.868966 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce4600-8e55-4456-9649-003555e83c1a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:46.928826 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7cce4600-8e55-4456-9649-003555e83c1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ea7368e39f343b68eb92c3b6a5f730a', 'request_id': 'req-7cce4600-8e55-4456-9649-003555e83c1a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:46.936594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7cce4600-8e55-4456-9649-003555e83c1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ea7368e39f343b68eb92c3b6a5f730a', 'request_id': 'req-7cce4600-8e55-4456-9649-003555e83c1a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:46.945318 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7cce4600-8e55-4456-9649-003555e83c1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ea7368e39f343b68eb92c3b6a5f730a', 'request_id': 'req-7cce4600-8e55-4456-9649-003555e83c1a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:46.960159 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7cce4600-8e55-4456-9649-003555e83c1a admin admin] The token cache is being invalidate because project 0ea7368e39f343b68eb92c3b6a5f730a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:43:46.960715 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7cce4600-8e55-4456-9649-003555e83c1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:46.961580 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7cce4600-8e55-4456-9649-003555e83c1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:46.963190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1325/2638] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:43:46 2022] DELETE /identity/v3/projects/0ea7368e39f343b68eb92c3b6a5f730a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:46.967900 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f2f39-7e9b-47cc-81d0-ebaf09707bfc admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:46.969462 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7c9f2f39-7e9b-47cc-81d0-ebaf09707bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:46.990371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f2f39-7e9b-47cc-81d0-ebaf09707bfc admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:47.061169 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfe578-475b-4cfa-9621-f84d3f02b0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:47.062284 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfe578-475b-4cfa-9621-f84d3f02b0b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:47.062783 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfe578-475b-4cfa-9621-f84d3f02b0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:47.063140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfe578-475b-4cfa-9621-f84d3f02b0b8 admin admin] PATH_INFO: `/v3/users/436f85bfb55a4d1a87b38a9ccc4e8903` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:47.064549 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfe578-475b-4cfa-9621-f84d3f02b0b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=436f85bfb55a4d1a87b38a9ccc4e8903)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:47.066537 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfe578-475b-4cfa-9621-f84d3f02b0b8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:47.121064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-80dfe578-475b-4cfa-9621-f84d3f02b0b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '436f85bfb55a4d1a87b38a9ccc4e8903'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:47.134903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-80dfe578-475b-4cfa-9621-f84d3f02b0b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '436f85bfb55a4d1a87b38a9ccc4e8903'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:47.144544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-80dfe578-475b-4cfa-9621-f84d3f02b0b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '436f85bfb55a4d1a87b38a9ccc4e8903'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:47.145247 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-80dfe578-475b-4cfa-9621-f84d3f02b0b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '436f85bfb55a4d1a87b38a9ccc4e8903'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:47.145874 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-80dfe578-475b-4cfa-9621-f84d3f02b0b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '436f85bfb55a4d1a87b38a9ccc4e8903'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:47.171013 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1314/2639] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:43:46 2022] DELETE /identity/v3/users/436f85bfb55a4d1a87b38a9ccc4e8903 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:47.457781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce4600-8e55-4456-9649-003555e83c1a admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:47.459404 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7cce4600-8e55-4456-9649-003555e83c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:47.473100 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce4600-8e55-4456-9649-003555e83c1a admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:47.534829 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4d8ad-4d9e-4225-9540-b08cf5ed5e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:47.535727 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4d8ad-4d9e-4225-9540-b08cf5ed5e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:47.536014 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4d8ad-4d9e-4225-9540-b08cf5ed5e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:47.536295 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4d8ad-4d9e-4225-9540-b08cf5ed5e7a admin admin] PATH_INFO: `/v3/projects/ce6d99420d6c4b349945e6ead1af603a` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:47.537472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4d8ad-4d9e-4225-9540-b08cf5ed5e7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce6d99420d6c4b349945e6ead1af603a)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:47.539410 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4d8ad-4d9e-4225-9540-b08cf5ed5e7a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:47.541888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:47.586639 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c2b4d8ad-4d9e-4225-9540-b08cf5ed5e7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce6d99420d6c4b349945e6ead1af603a', 'request_id': 'req-c2b4d8ad-4d9e-4225-9540-b08cf5ed5e7a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:47.593865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c2b4d8ad-4d9e-4225-9540-b08cf5ed5e7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce6d99420d6c4b349945e6ead1af603a', 'request_id': 'req-c2b4d8ad-4d9e-4225-9540-b08cf5ed5e7a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:47.600509 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c2b4d8ad-4d9e-4225-9540-b08cf5ed5e7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce6d99420d6c4b349945e6ead1af603a', 'request_id': 'req-c2b4d8ad-4d9e-4225-9540-b08cf5ed5e7a'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:47.611123 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c2b4d8ad-4d9e-4225-9540-b08cf5ed5e7a admin admin] The token cache is being invalidate because project ce6d99420d6c4b349945e6ead1af603a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:43:47.611418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c2b4d8ad-4d9e-4225-9540-b08cf5ed5e7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:47.612047 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c2b4d8ad-4d9e-4225-9540-b08cf5ed5e7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:47.613564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1326/2640] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:43:47 2022] DELETE /identity/v3/projects/ce6d99420d6c4b349945e6ead1af603a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:47.619611 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb0d39-fdf3-40a4-b449-9ef3a5eaa94b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:47.620103 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb0d39-fdf3-40a4-b449-9ef3a5eaa94b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:47.620208 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb0d39-fdf3-40a4-b449-9ef3a5eaa94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:47.640981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-61eb0d39-fdf3-40a4-b449-9ef3a5eaa94b None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:47.642123 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-61eb0d39-fdf3-40a4-b449-9ef3a5eaa94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:47.661557 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:47.703873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1315/2641] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:47.713475 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19927b83-3433-44a1-85bc-b0963d56399c None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:47.713807 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19927b83-3433-44a1-85bc-b0963d56399c None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:47.714080 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19927b83-3433-44a1-85bc-b0963d56399c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:47.733692 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-19927b83-3433-44a1-85bc-b0963d56399c None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:47.734886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-19927b83-3433-44a1-85bc-b0963d56399c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:47.742951 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1327/2642] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:47.747691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb0d39-fdf3-40a4-b449-9ef3a5eaa94b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:47.757367 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb0d39-fdf3-40a4-b449-9ef3a5eaa94b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:47.761984 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a2e62-b928-4237-b264-fcfa5ad64cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:47.763347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a2e62-b928-4237-b264-fcfa5ad64cad admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:47.763656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a2e62-b928-4237-b264-fcfa5ad64cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:47.763943 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a2e62-b928-4237-b264-fcfa5ad64cad admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:47.764669 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a2e62-b928-4237-b264-fcfa5ad64cad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:43:47.764983 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a2e62-b928-4237-b264-fcfa5ad64cad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:47.766575 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a2e62-b928-4237-b264-fcfa5ad64cad admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:47.779185 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1316/2643] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:43:47.783656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19927b83-3433-44a1-85bc-b0963d56399c None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:47.793219 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19927b83-3433-44a1-85bc-b0963d56399c None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:47.798393 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11656a-8c77-48af-8aa3-9d8b53f35849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:47.799092 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11656a-8c77-48af-8aa3-9d8b53f35849 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:47.799384 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11656a-8c77-48af-8aa3-9d8b53f35849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:47.799669 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11656a-8c77-48af-8aa3-9d8b53f35849 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:47.800366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11656a-8c77-48af-8aa3-9d8b53f35849 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:47.802302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11656a-8c77-48af-8aa3-9d8b53f35849 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:47.828226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1328/2644] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:43:47 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:47.832304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a2e62-b928-4237-b264-fcfa5ad64cad admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:47.842295 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a2e62-b928-4237-b264-fcfa5ad64cad admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:47.901927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5b2cf-79c8-4d8f-bf83-16265d2e2018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:47.902527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5b2cf-79c8-4d8f-bf83-16265d2e2018 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:47.902818 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5b2cf-79c8-4d8f-bf83-16265d2e2018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:47.903146 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5b2cf-79c8-4d8f-bf83-16265d2e2018 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:47.904469 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5b2cf-79c8-4d8f-bf83-16265d2e2018 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:47.906441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5b2cf-79c8-4d8f-bf83-16265d2e2018 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:47.928465 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1317/2645] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:43:47 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:47.932465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11656a-8c77-48af-8aa3-9d8b53f35849 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:47.943634 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11656a-8c77-48af-8aa3-9d8b53f35849 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:47.949583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7719ed-7a9d-4c8e-b310-d1e473bb4a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:47.950313 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7719ed-7a9d-4c8e-b310-d1e473bb4a78 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:47.950598 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7719ed-7a9d-4c8e-b310-d1e473bb4a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:47.951102 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7719ed-7a9d-4c8e-b310-d1e473bb4a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:47.951847 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7719ed-7a9d-4c8e-b310-d1e473bb4a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:47.953471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7719ed-7a9d-4c8e-b310-d1e473bb4a78 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:47.967886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1329/2646] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:47.969466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5b2cf-79c8-4d8f-bf83-16265d2e2018 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:47.979402 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5b2cf-79c8-4d8f-bf83-16265d2e2018 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:47.984775 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205889e0-3fe0-42e8-918c-c4a99f63a941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:47.985788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205889e0-3fe0-42e8-918c-c4a99f63a941 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:47.986111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205889e0-3fe0-42e8-918c-c4a99f63a941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:47.986511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205889e0-3fe0-42e8-918c-c4a99f63a941 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:47.987417 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205889e0-3fe0-42e8-918c-c4a99f63a941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:47.989259 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205889e0-3fe0-42e8-918c-c4a99f63a941 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:48.000644 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1318/2647] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:48.004468 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7719ed-7a9d-4c8e-b310-d1e473bb4a78 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:48.016347 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7719ed-7a9d-4c8e-b310-d1e473bb4a78 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:48.021044 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07de522-148f-4e54-8522-14e899c5d281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:48.021939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07de522-148f-4e54-8522-14e899c5d281 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:48.022223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07de522-148f-4e54-8522-14e899c5d281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:48.022501 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07de522-148f-4e54-8522-14e899c5d281 admin admin] PATH_INFO: `/v3/projects/7fda95efa0af4164902ba45a42669d77/users/b4eaffb133fd45c8a99667b06e0e849f/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:48.038570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07de522-148f-4e54-8522-14e899c5d281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fda95efa0af4164902ba45a42669d77, user_id=b4eaffb133fd45c8a99667b06e0e849f, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:48.041236 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07de522-148f-4e54-8522-14e899c5d281 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:48.057149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1330/2648] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:48 2022] PUT /identity/v3/projects/7fda95efa0af4164902ba45a42669d77/users/b4eaffb133fd45c8a99667b06e0e849f/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:43:48.068700 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767ed79-18da-4a32-b928-36a521a5b765 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:48.068994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767ed79-18da-4a32-b928-36a521a5b765 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:48.069280 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767ed79-18da-4a32-b928-36a521a5b765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:48.097461 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-5767ed79-18da-4a32-b928-36a521a5b765 None None] MFA Rules not processed for user `b4eaffb133fd45c8a99667b06e0e849f`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:48.098780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5767ed79-18da-4a32-b928-36a521a5b765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:48.176658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1319/2649] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:43:48.181321 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07de522-148f-4e54-8522-14e899c5d281 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:48.244084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07de522-148f-4e54-8522-14e899c5d281 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:48.249021 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7dca7c-5c0b-4a28-b358-94def898211b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:48.249686 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7dca7c-5c0b-4a28-b358-94def898211b admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:48.249971 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7dca7c-5c0b-4a28-b358-94def898211b admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:48.250246 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7dca7c-5c0b-4a28-b358-94def898211b admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:48.251015 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7dca7c-5c0b-4a28-b358-94def898211b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:48.252848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7dca7c-5c0b-4a28-b358-94def898211b admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:48.278215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1331/2650] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:43:48 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:43:48.283314 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767ed79-18da-4a32-b928-36a521a5b765 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:48.294822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767ed79-18da-4a32-b928-36a521a5b765 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:48.311927 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:48.356239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02d720-fece-498d-a669-fe4e827d4b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:48.356966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02d720-fece-498d-a669-fe4e827d4b01 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:48.357230 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02d720-fece-498d-a669-fe4e827d4b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:48.357530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02d720-fece-498d-a669-fe4e827d4b01 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:48.358226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02d720-fece-498d-a669-fe4e827d4b01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:48.360242 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02d720-fece-498d-a669-fe4e827d4b01 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:48.383772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1320/2651] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:43:48 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:43:48.388827 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7dca7c-5c0b-4a28-b358-94def898211b admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:48.399075 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7dca7c-5c0b-4a28-b358-94def898211b admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:48.405967 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11563a8-54c2-446e-ab86-11718f1db0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:48.406651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11563a8-54c2-446e-ab86-11718f1db0bc admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:48.406969 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11563a8-54c2-446e-ab86-11718f1db0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:48.407255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11563a8-54c2-446e-ab86-11718f1db0bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:48.407992 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11563a8-54c2-446e-ab86-11718f1db0bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:48.409636 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11563a8-54c2-446e-ab86-11718f1db0bc admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:48.421374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1332/2652] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:48.425470 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02d720-fece-498d-a669-fe4e827d4b01 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:48.436741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02d720-fece-498d-a669-fe4e827d4b01 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:48.441572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb40522-92f9-4a10-b336-2434262aeb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:48.442506 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb40522-92f9-4a10-b336-2434262aeb3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:48.442796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb40522-92f9-4a10-b336-2434262aeb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:48.443113 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb40522-92f9-4a10-b336-2434262aeb3f admin admin] PATH_INFO: `/v3/projects/0186183db3784362879d2bcacfdcf248/users/f85233a42e8e4e53a56e34773f8c3390/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:48.459044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb40522-92f9-4a10-b336-2434262aeb3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0186183db3784362879d2bcacfdcf248, user_id=f85233a42e8e4e53a56e34773f8c3390, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:48.461716 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb40522-92f9-4a10-b336-2434262aeb3f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:48.477110 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1321/2653] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:48 2022] PUT /identity/v3/projects/0186183db3784362879d2bcacfdcf248/users/f85233a42e8e4e53a56e34773f8c3390/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:48.481162 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11563a8-54c2-446e-ab86-11718f1db0bc admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:48.486564 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:48.493335 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11563a8-54c2-446e-ab86-11718f1db0bc admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:48.551064 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db77dd-c0f9-44f4-939f-342bddf8d69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:48.551729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db77dd-c0f9-44f4-939f-342bddf8d69a admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:48.552017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db77dd-c0f9-44f4-939f-342bddf8d69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:48.552292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db77dd-c0f9-44f4-939f-342bddf8d69a admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:48.553000 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db77dd-c0f9-44f4-939f-342bddf8d69a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:48.554585 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db77dd-c0f9-44f4-939f-342bddf8d69a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:48.564851 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1333/2654] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:43:48.569117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb40522-92f9-4a10-b336-2434262aeb3f admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:48.579992 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb40522-92f9-4a10-b336-2434262aeb3f admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:48.584613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28593d4f-4d40-493c-89fc-f9a400d12e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:48.585488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28593d4f-4d40-493c-89fc-f9a400d12e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:48.585772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28593d4f-4d40-493c-89fc-f9a400d12e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:48.586053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28593d4f-4d40-493c-89fc-f9a400d12e5e admin admin] PATH_INFO: `/v3/projects/0186183db3784362879d2bcacfdcf248/users/f85233a42e8e4e53a56e34773f8c3390/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:48.588042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28593d4f-4d40-493c-89fc-f9a400d12e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0186183db3784362879d2bcacfdcf248, user_id=f85233a42e8e4e53a56e34773f8c3390, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:48.590623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28593d4f-4d40-493c-89fc-f9a400d12e5e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:48.600573 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1322/2655] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:43:48 2022] PUT /identity/v3/projects/0186183db3784362879d2bcacfdcf248/users/f85233a42e8e4e53a56e34773f8c3390/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:48.611077 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870b77b-2426-4d62-beb4-4a9b493db3c6 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:48.611375 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870b77b-2426-4d62-beb4-4a9b493db3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:48.611655 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870b77b-2426-4d62-beb4-4a9b493db3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:48.638223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-9870b77b-2426-4d62-beb4-4a9b493db3c6 None None] MFA Rules not processed for user `f85233a42e8e4e53a56e34773f8c3390`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:48.639568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-9870b77b-2426-4d62-beb4-4a9b493db3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:48.705471 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1334/2656] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:43:48.716673 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28593d4f-4d40-493c-89fc-f9a400d12e5e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:48.718445 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-28593d4f-4d40-493c-89fc-f9a400d12e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:48.775704 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28593d4f-4d40-493c-89fc-f9a400d12e5e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:48.779404 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a0684-8f92-453c-a872-6b90044326d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:48.780044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a0684-8f92-453c-a872-6b90044326d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:48.780347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a0684-8f92-453c-a872-6b90044326d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:48.780605 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a0684-8f92-453c-a872-6b90044326d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:48.835818 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a0684-8f92-453c-a872-6b90044326d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:48.837244 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a0684-8f92-453c-a872-6b90044326d1 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:48.845957 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1323/2657] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:51.052757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6971f1-f76f-4b1e-8f2d-9666fb0f57b7 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:51.053327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6971f1-f76f-4b1e-8f2d-9666fb0f57b7 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:51.053327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6971f1-f76f-4b1e-8f2d-9666fb0f57b7 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:51.054673 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1335/2658] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 22:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:43:51.060678 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cdff8-7819-4721-896c-d5aa66da4b78 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:51.061167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cdff8-7819-4721-896c-d5aa66da4b78 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:51.061269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cdff8-7819-4721-896c-d5aa66da4b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:51.083053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-097cdff8-7819-4721-896c-d5aa66da4b78 None None] MFA Rules not processed for user `35d686923ef541819de3c77847d38511`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:51.084325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-097cdff8-7819-4721-896c-d5aa66da4b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:51.106044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:51.159046 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1324/2659] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 22:43:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:43:51.170508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6971f1-f76f-4b1e-8f2d-9666fb0f57b7 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:51.172272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-4b6971f1-f76f-4b1e-8f2d-9666fb0f57b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:51.250092 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6971f1-f76f-4b1e-8f2d-9666fb0f57b7 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:51.253714 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b35c83-58eb-4add-95e2-5255b3d0abc1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36de21e0b3de48dabe0f27a9db8a37aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:51.254367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b35c83-58eb-4add-95e2-5255b3d0abc1 service glance] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:51.254659 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b35c83-58eb-4add-95e2-5255b3d0abc1 service glance] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:51.254983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b35c83-58eb-4add-95e2-5255b3d0abc1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:51.265570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b35c83-58eb-4add-95e2-5255b3d0abc1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:51.267038 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b35c83-58eb-4add-95e2-5255b3d0abc1 service glance] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:51.277617 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1336/2660] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:43:51.345462 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa213e-0528-4b2a-82de-bcb08fa11eba None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:51.346048 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa213e-0528-4b2a-82de-bcb08fa11eba None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:51.346339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa213e-0528-4b2a-82de-bcb08fa11eba None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:51.347770 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1325/2661] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 22:43:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:43:51.352613 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0a73d-10b8-4440-a0b0-66732be70537 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:51.352903 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0a73d-10b8-4440-a0b0-66732be70537 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:51.353182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0a73d-10b8-4440-a0b0-66732be70537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:51.366370 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:51.372841 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-22b0a73d-10b8-4440-a0b0-66732be70537 None None] MFA Rules not processed for user `37ba527b96634fc2a48b6ca9597d8191`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:51.374089 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-22b0a73d-10b8-4440-a0b0-66732be70537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:51.425492 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1337/2662] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 22:43:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:51.437588 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa213e-0528-4b2a-82de-bcb08fa11eba None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:51.439350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8cfa213e-0528-4b2a-82de-bcb08fa11eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:51.453130 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa213e-0528-4b2a-82de-bcb08fa11eba None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:51.456732 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a21cbe-4b2a-4b5a-9d36-2802424a6aa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:51.457370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a21cbe-4b2a-4b5a-9d36-2802424a6aa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:51.457660 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a21cbe-4b2a-4b5a-9d36-2802424a6aa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:51.457944 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a21cbe-4b2a-4b5a-9d36-2802424a6aa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:51.464918 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a21cbe-4b2a-4b5a-9d36-2802424a6aa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:51.466428 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a21cbe-4b2a-4b5a-9d36-2802424a6aa8 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:51.474223 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1326/2663] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:51.555223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132f157-0cfd-4090-9e89-ba106f965c94 None None] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:51.555551 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132f157-0cfd-4090-9e89-ba106f965c94 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:51.555848 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132f157-0cfd-4090-9e89-ba106f965c94 None None] PATH_INFO: `/` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:51.557226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1338/2664] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 22:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:43:51.563061 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f49ac-e82b-410b-b7ad-d32f0bdc8b4b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:51.563351 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f49ac-e82b-410b-b7ad-d32f0bdc8b4b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:51.563630 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f49ac-e82b-410b-b7ad-d32f0bdc8b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:51.582336 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-669f49ac-e82b-410b-b7ad-d32f0bdc8b4b None None] MFA Rules not processed for user `35d686923ef541819de3c77847d38511`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:51.583559 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-669f49ac-e82b-410b-b7ad-d32f0bdc8b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:51.593126 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1327/2665] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 22:43:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:43:51.603620 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132f157-0cfd-4090-9e89-ba106f965c94 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:51.605265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-5132f157-0cfd-4090-9e89-ba106f965c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:51.619035 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132f157-0cfd-4090-9e89-ba106f965c94 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:51.623220 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0cbaf-42a5-48bd-be1a-3d8ff1ab2f72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:51.623861 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0cbaf-42a5-48bd-be1a-3d8ff1ab2f72 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:51.624146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0cbaf-42a5-48bd-be1a-3d8ff1ab2f72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:51.624428 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0cbaf-42a5-48bd-be1a-3d8ff1ab2f72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:51.634618 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0cbaf-42a5-48bd-be1a-3d8ff1ab2f72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:51.636132 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0cbaf-42a5-48bd-be1a-3d8ff1ab2f72 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:51.647990 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1339/2666] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:53.080993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f49ac-e82b-410b-b7ad-d32f0bdc8b4b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:53.082952 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-669f49ac-e82b-410b-b7ad-d32f0bdc8b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:53.095453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f49ac-e82b-410b-b7ad-d32f0bdc8b4b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:53.099298 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0658be-a409-4de8-80c7-03684437a3e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:53.099973 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0658be-a409-4de8-80c7-03684437a3e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:53.100261 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0658be-a409-4de8-80c7-03684437a3e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:53.100551 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0658be-a409-4de8-80c7-03684437a3e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:53.110427 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0658be-a409-4de8-80c7-03684437a3e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:53.111886 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0658be-a409-4de8-80c7-03684437a3e6 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:53.123897 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1328/2667] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:54.136050 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d40e32-28bb-4015-a19f-20451f862b48 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:54.136518 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d40e32-28bb-4015-a19f-20451f862b48 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:54.136626 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d40e32-28bb-4015-a19f-20451f862b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:54.155328 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-01d40e32-28bb-4015-a19f-20451f862b48 None None] MFA Rules not processed for user `df1aeb2be41c49819b811ecfa47d4155`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:54.156514 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-01d40e32-28bb-4015-a19f-20451f862b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:54.209781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1340/2668] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 22:43:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:43:54.222102 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0658be-a409-4de8-80c7-03684437a3e6 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:54.223542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8a0658be-a409-4de8-80c7-03684437a3e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:54.290074 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0658be-a409-4de8-80c7-03684437a3e6 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:54.295572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaed353-2f3e-4944-8011-cc2ab958f4a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ce35c43e8294f4cbc880f2219a6eb76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:54.296212 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaed353-2f3e-4944-8011-cc2ab958f4a3 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:54.296502 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaed353-2f3e-4944-8011-cc2ab958f4a3 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:54.296785 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaed353-2f3e-4944-8011-cc2ab958f4a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:54.304820 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaed353-2f3e-4944-8011-cc2ab958f4a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:54.306396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaed353-2f3e-4944-8011-cc2ab958f4a3 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:54.314359 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1329/2669] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 22:43:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:43:58.348585 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d40e32-28bb-4015-a19f-20451f862b48 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:58.439833 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d40e32-28bb-4015-a19f-20451f862b48 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:58.446531 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fae9a0-2de4-4aa1-b788-97d1f1d0d4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:58.447449 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fae9a0-2de4-4aa1-b788-97d1f1d0d4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:58.447740 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fae9a0-2de4-4aa1-b788-97d1f1d0d4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:58.448021 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fae9a0-2de4-4aa1-b788-97d1f1d0d4dc admin admin] PATH_INFO: `/v3/users/b4eaffb133fd45c8a99667b06e0e849f` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:58.449421 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fae9a0-2de4-4aa1-b788-97d1f1d0d4dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4eaffb133fd45c8a99667b06e0e849f)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:58.452157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fae9a0-2de4-4aa1-b788-97d1f1d0d4dc admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:58.552668 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b8fae9a0-2de4-4aa1-b788-97d1f1d0d4dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4eaffb133fd45c8a99667b06e0e849f'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:58.554850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:58.572104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b8fae9a0-2de4-4aa1-b788-97d1f1d0d4dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4eaffb133fd45c8a99667b06e0e849f'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:58.592176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b8fae9a0-2de4-4aa1-b788-97d1f1d0d4dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4eaffb133fd45c8a99667b06e0e849f'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:58.602699 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b8fae9a0-2de4-4aa1-b788-97d1f1d0d4dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4eaffb133fd45c8a99667b06e0e849f'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:58.603706 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b8fae9a0-2de4-4aa1-b788-97d1f1d0d4dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4eaffb133fd45c8a99667b06e0e849f'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:58.611157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1341/2670] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:43:58 2022] DELETE /identity/v3/users/b4eaffb133fd45c8a99667b06e0e849f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:43:58.615819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb46d9-3668-4a1c-9110-fa3ad2415d7a None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:58.616350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb46d9-3668-4a1c-9110-fa3ad2415d7a None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:58.616451 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb46d9-3668-4a1c-9110-fa3ad2415d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:58.637396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-3acb46d9-3668-4a1c-9110-fa3ad2415d7a None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:43:58.638807 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3acb46d9-3668-4a1c-9110-fa3ad2415d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:58.705590 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1330/2671] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:43:58.716112 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fae9a0-2de4-4aa1-b788-97d1f1d0d4dc admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:58.718095 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b8fae9a0-2de4-4aa1-b788-97d1f1d0d4dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:58.786403 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fae9a0-2de4-4aa1-b788-97d1f1d0d4dc admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:58.790573 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc77f1-b87a-457b-90e6-701a624bd45f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:58.791321 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc77f1-b87a-457b-90e6-701a624bd45f service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:58.791634 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc77f1-b87a-457b-90e6-701a624bd45f service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:58.792224 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc77f1-b87a-457b-90e6-701a624bd45f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:58.807743 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc77f1-b87a-457b-90e6-701a624bd45f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:58.810398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc77f1-b87a-457b-90e6-701a624bd45f service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:58.827197 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1342/2672] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:43:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:43:59.151479 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb46d9-3668-4a1c-9110-fa3ad2415d7a None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:59.153059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3acb46d9-3668-4a1c-9110-fa3ad2415d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:43:59.175665 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb46d9-3668-4a1c-9110-fa3ad2415d7a None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:59.182272 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475c375-f18c-4230-83ee-de8129ffd1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:59.192656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475c375-f18c-4230-83ee-de8129ffd1f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:59.193061 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475c375-f18c-4230-83ee-de8129ffd1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:59.193358 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475c375-f18c-4230-83ee-de8129ffd1f9 admin admin] PATH_INFO: `/v3/projects/7fda95efa0af4164902ba45a42669d77` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:59.195296 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475c375-f18c-4230-83ee-de8129ffd1f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fda95efa0af4164902ba45a42669d77)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:59.197402 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475c375-f18c-4230-83ee-de8129ffd1f9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:59.210191 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:59.257016 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4475c375-f18c-4230-83ee-de8129ffd1f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fda95efa0af4164902ba45a42669d77', 'request_id': 'req-4475c375-f18c-4230-83ee-de8129ffd1f9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:59.265932 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4475c375-f18c-4230-83ee-de8129ffd1f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fda95efa0af4164902ba45a42669d77', 'request_id': 'req-4475c375-f18c-4230-83ee-de8129ffd1f9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:59.272309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4475c375-f18c-4230-83ee-de8129ffd1f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fda95efa0af4164902ba45a42669d77', 'request_id': 'req-4475c375-f18c-4230-83ee-de8129ffd1f9'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:59.286545 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4475c375-f18c-4230-83ee-de8129ffd1f9 admin admin] The token cache is being invalidate because project 7fda95efa0af4164902ba45a42669d77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:43:59.286841 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4475c375-f18c-4230-83ee-de8129ffd1f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:59.287556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-4475c375-f18c-4230-83ee-de8129ffd1f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:59.289148 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1331/2673] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:43:59 2022] DELETE /identity/v3/projects/7fda95efa0af4164902ba45a42669d77 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:43:59.293583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc77f1-b87a-457b-90e6-701a624bd45f service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:43:59.304056 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc77f1-b87a-457b-90e6-701a624bd45f service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:43:59.375721 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fabc23-1fa2-4efe-8ad0-af016b1c6881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:43:59.377694 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fabc23-1fa2-4efe-8ad0-af016b1c6881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:43:59.377694 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fabc23-1fa2-4efe-8ad0-af016b1c6881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:43:59.378057 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fabc23-1fa2-4efe-8ad0-af016b1c6881 admin admin] PATH_INFO: `/v3/users/f85233a42e8e4e53a56e34773f8c3390` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:43:59.382671 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fabc23-1fa2-4efe-8ad0-af016b1c6881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f85233a42e8e4e53a56e34773f8c3390)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:43:59.385974 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fabc23-1fa2-4efe-8ad0-af016b1c6881 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:43:59.579255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d7fabc23-1fa2-4efe-8ad0-af016b1c6881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f85233a42e8e4e53a56e34773f8c3390'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:59.596656 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d7fabc23-1fa2-4efe-8ad0-af016b1c6881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f85233a42e8e4e53a56e34773f8c3390'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:59.614897 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d7fabc23-1fa2-4efe-8ad0-af016b1c6881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f85233a42e8e4e53a56e34773f8c3390'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:59.619855 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:43:59.636428 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d7fabc23-1fa2-4efe-8ad0-af016b1c6881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f85233a42e8e4e53a56e34773f8c3390'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:59.637384 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-d7fabc23-1fa2-4efe-8ad0-af016b1c6881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f85233a42e8e4e53a56e34773f8c3390'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:43:59.648221 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1343/2674] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:43:59 2022] DELETE /identity/v3/users/f85233a42e8e4e53a56e34773f8c3390 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:44:00.139226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475c375-f18c-4230-83ee-de8129ffd1f9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:00.142367 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4475c375-f18c-4230-83ee-de8129ffd1f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:00.161039 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475c375-f18c-4230-83ee-de8129ffd1f9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:00.269706 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce26b0-5c7a-4164-a8b4-729e6ae8ea9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:00.270849 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce26b0-5c7a-4164-a8b4-729e6ae8ea9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:00.271225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce26b0-5c7a-4164-a8b4-729e6ae8ea9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:00.271522 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce26b0-5c7a-4164-a8b4-729e6ae8ea9f admin admin] PATH_INFO: `/v3/projects/0186183db3784362879d2bcacfdcf248` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:00.272776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce26b0-5c7a-4164-a8b4-729e6ae8ea9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0186183db3784362879d2bcacfdcf248)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:00.274713 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce26b0-5c7a-4164-a8b4-729e6ae8ea9f admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:00.333832 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-74ce26b0-5c7a-4164-a8b4-729e6ae8ea9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0186183db3784362879d2bcacfdcf248', 'request_id': 'req-74ce26b0-5c7a-4164-a8b4-729e6ae8ea9f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:00.344287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-74ce26b0-5c7a-4164-a8b4-729e6ae8ea9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0186183db3784362879d2bcacfdcf248', 'request_id': 'req-74ce26b0-5c7a-4164-a8b4-729e6ae8ea9f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:00.351405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-74ce26b0-5c7a-4164-a8b4-729e6ae8ea9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0186183db3784362879d2bcacfdcf248', 'request_id': 'req-74ce26b0-5c7a-4164-a8b4-729e6ae8ea9f'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:00.376820 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-74ce26b0-5c7a-4164-a8b4-729e6ae8ea9f admin admin] The token cache is being invalidate because project 0186183db3784362879d2bcacfdcf248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:44:00.377371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-74ce26b0-5c7a-4164-a8b4-729e6ae8ea9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:00.378134 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-74ce26b0-5c7a-4164-a8b4-729e6ae8ea9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:00.381202 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1332/2675] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:44:00 2022] DELETE /identity/v3/projects/0186183db3784362879d2bcacfdcf248 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:44:00.389086 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b1c5c-0c45-44eb-81d5-e7d4af18fbf9 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:00.389798 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b1c5c-0c45-44eb-81d5-e7d4af18fbf9 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:00.389798 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b1c5c-0c45-44eb-81d5-e7d4af18fbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:00.420050 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-301b1c5c-0c45-44eb-81d5-e7d4af18fbf9 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:44:00.421664 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-301b1c5c-0c45-44eb-81d5-e7d4af18fbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:00.520972 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1344/2676] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:44:00.531856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4bb73-8e06-4697-9a27-9c69441aa121 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:00.532119 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4bb73-8e06-4697-9a27-9c69441aa121 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:00.532413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4bb73-8e06-4697-9a27-9c69441aa121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:00.560337 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-01d4bb73-8e06-4697-9a27-9c69441aa121 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:44:00.561876 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-01d4bb73-8e06-4697-9a27-9c69441aa121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:00.571879 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1333/2677] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:44:00.576745 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b1c5c-0c45-44eb-81d5-e7d4af18fbf9 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:00.588983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b1c5c-0c45-44eb-81d5-e7d4af18fbf9 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:00.594545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3655e-a441-4c2d-bbd1-b2a17d70abc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:00.595260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3655e-a441-4c2d-bbd1-b2a17d70abc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:00.595545 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3655e-a441-4c2d-bbd1-b2a17d70abc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:00.595823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3655e-a441-4c2d-bbd1-b2a17d70abc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:00.596552 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3655e-a441-4c2d-bbd1-b2a17d70abc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:44:00.596866 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3655e-a441-4c2d-bbd1-b2a17d70abc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:00.598671 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3655e-a441-4c2d-bbd1-b2a17d70abc0 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:00.617351 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1345/2678] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:44:00.622515 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4bb73-8e06-4697-9a27-9c69441aa121 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:00.632556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4bb73-8e06-4697-9a27-9c69441aa121 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:00.638497 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c9329-f130-497b-a186-cab7c3c9364c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:00.639316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c9329-f130-497b-a186-cab7c3c9364c admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:00.639607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c9329-f130-497b-a186-cab7c3c9364c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:00.639977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c9329-f130-497b-a186-cab7c3c9364c admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:00.640842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c9329-f130-497b-a186-cab7c3c9364c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:00.643412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c9329-f130-497b-a186-cab7c3c9364c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:00.666454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:44:00.677972 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1334/2679] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:44:00 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:44:00.682438 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3655e-a441-4c2d-bbd1-b2a17d70abc0 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:00.693658 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3655e-a441-4c2d-bbd1-b2a17d70abc0 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:00.757048 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f5b9a-259d-42e6-a4aa-a1cdf4b5c61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:00.757762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f5b9a-259d-42e6-a4aa-a1cdf4b5c61e admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:00.758056 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f5b9a-259d-42e6-a4aa-a1cdf4b5c61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:00.758341 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f5b9a-259d-42e6-a4aa-a1cdf4b5c61e admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:00.759096 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f5b9a-259d-42e6-a4aa-a1cdf4b5c61e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:00.761017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f5b9a-259d-42e6-a4aa-a1cdf4b5c61e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:00.784575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1346/2680] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:44:00 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:44:00.788632 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c9329-f130-497b-a186-cab7c3c9364c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:00.800626 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c9329-f130-497b-a186-cab7c3c9364c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:00.805658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9815542-7f77-4d9e-bc40-06fbde8cbd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:00.806334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9815542-7f77-4d9e-bc40-06fbde8cbd00 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:00.806620 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9815542-7f77-4d9e-bc40-06fbde8cbd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:00.806901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9815542-7f77-4d9e-bc40-06fbde8cbd00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:00.807668 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9815542-7f77-4d9e-bc40-06fbde8cbd00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:00.809272 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9815542-7f77-4d9e-bc40-06fbde8cbd00 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:00.824447 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1335/2681] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:44:00.825679 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f5b9a-259d-42e6-a4aa-a1cdf4b5c61e admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:00.836024 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f5b9a-259d-42e6-a4aa-a1cdf4b5c61e admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:00.846963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecf114-e756-492b-a46c-54fe91fed523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:00.846963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecf114-e756-492b-a46c-54fe91fed523 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:00.846963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecf114-e756-492b-a46c-54fe91fed523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:00.846963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecf114-e756-492b-a46c-54fe91fed523 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:00.847452 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ecf114-e756-492b-a46c-54fe91fed523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:00.847452 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ecf114-e756-492b-a46c-54fe91fed523 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:00.862988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1347/2682] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:44:00.863977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9815542-7f77-4d9e-bc40-06fbde8cbd00 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:00.881555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9815542-7f77-4d9e-bc40-06fbde8cbd00 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:00.887224 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203632aa-7671-4a79-8c1c-1537533d8050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:00.888161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203632aa-7671-4a79-8c1c-1537533d8050 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:00.888448 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203632aa-7671-4a79-8c1c-1537533d8050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:00.888728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203632aa-7671-4a79-8c1c-1537533d8050 admin admin] PATH_INFO: `/v3/projects/bea3d94bf55241dcab0f0896f77abc4c/users/f3b88266142043d89bf0ba8ee1372f1c/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:00.905284 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203632aa-7671-4a79-8c1c-1537533d8050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bea3d94bf55241dcab0f0896f77abc4c, user_id=f3b88266142043d89bf0ba8ee1372f1c, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:00.913161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203632aa-7671-4a79-8c1c-1537533d8050 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:00.924455 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1336/2683] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:44:00 2022] PUT /identity/v3/projects/bea3d94bf55241dcab0f0896f77abc4c/users/f3b88266142043d89bf0ba8ee1372f1c/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:44:00.936119 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9e28a-49f7-45d6-9306-a61af32d5f16 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:00.936505 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9e28a-49f7-45d6-9306-a61af32d5f16 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:00.936859 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9e28a-49f7-45d6-9306-a61af32d5f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:00.967456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-3ef9e28a-49f7-45d6-9306-a61af32d5f16 None None] MFA Rules not processed for user `f3b88266142043d89bf0ba8ee1372f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:44:00.969311 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3ef9e28a-49f7-45d6-9306-a61af32d5f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:01.030101 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:44:01.069819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1348/2684] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:44:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:44:01.082594 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203632aa-7671-4a79-8c1c-1537533d8050 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:01.084731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-203632aa-7671-4a79-8c1c-1537533d8050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:01.161771 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203632aa-7671-4a79-8c1c-1537533d8050 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:01.165375 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d1eb5-6014-46c2-871f-75977855c06d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:01.166035 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d1eb5-6014-46c2-871f-75977855c06d service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:01.166319 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d1eb5-6014-46c2-871f-75977855c06d service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:01.166610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d1eb5-6014-46c2-871f-75977855c06d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:01.176443 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d1eb5-6014-46c2-871f-75977855c06d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:01.177971 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d1eb5-6014-46c2-871f-75977855c06d service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:01.188129 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1337/2685] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:44:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:44:01.208884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9e28a-49f7-45d6-9306-a61af32d5f16 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:01.279885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9e28a-49f7-45d6-9306-a61af32d5f16 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:01.284453 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8db3f-f1b2-47ff-b2a5-b79c9467de65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:01.285364 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8db3f-f1b2-47ff-b2a5-b79c9467de65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:01.285653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8db3f-f1b2-47ff-b2a5-b79c9467de65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:01.285934 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8db3f-f1b2-47ff-b2a5-b79c9467de65 admin admin] PATH_INFO: `/v3/users/f3b88266142043d89bf0ba8ee1372f1c` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:01.287185 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8db3f-f1b2-47ff-b2a5-b79c9467de65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b88266142043d89bf0ba8ee1372f1c)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:01.289118 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8db3f-f1b2-47ff-b2a5-b79c9467de65 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:01.354183 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-2de8db3f-f1b2-47ff-b2a5-b79c9467de65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b88266142043d89bf0ba8ee1372f1c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:01.367809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-2de8db3f-f1b2-47ff-b2a5-b79c9467de65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b88266142043d89bf0ba8ee1372f1c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:01.375559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-2de8db3f-f1b2-47ff-b2a5-b79c9467de65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b88266142043d89bf0ba8ee1372f1c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:01.384916 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-2de8db3f-f1b2-47ff-b2a5-b79c9467de65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b88266142043d89bf0ba8ee1372f1c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:01.385659 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-2de8db3f-f1b2-47ff-b2a5-b79c9467de65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b88266142043d89bf0ba8ee1372f1c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:01.395368 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1349/2686] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:44:01 2022] DELETE /identity/v3/users/f3b88266142043d89bf0ba8ee1372f1c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:44:01.397813 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595491d-beb6-43ec-af16-9e5ab020a140 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:01.398114 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595491d-beb6-43ec-af16-9e5ab020a140 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:01.398422 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595491d-beb6-43ec-af16-9e5ab020a140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:01.421052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8595491d-beb6-43ec-af16-9e5ab020a140 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:44:01.422387 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8595491d-beb6-43ec-af16-9e5ab020a140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:01.493610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1338/2687] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:44:01.504409 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8db3f-f1b2-47ff-b2a5-b79c9467de65 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:01.505880 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-2de8db3f-f1b2-47ff-b2a5-b79c9467de65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:01.571645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8db3f-f1b2-47ff-b2a5-b79c9467de65 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:01.576435 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c6d28-2ccf-47cb-b579-e0b3a8d46aac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:01.577187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c6d28-2ccf-47cb-b579-e0b3a8d46aac service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:01.577477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c6d28-2ccf-47cb-b579-e0b3a8d46aac service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:01.577781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c6d28-2ccf-47cb-b579-e0b3a8d46aac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:01.588235 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c6d28-2ccf-47cb-b579-e0b3a8d46aac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:01.590143 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c6d28-2ccf-47cb-b579-e0b3a8d46aac service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:01.601131 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1350/2688] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:44:01.900011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595491d-beb6-43ec-af16-9e5ab020a140 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:01.901779 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8595491d-beb6-43ec-af16-9e5ab020a140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:01.914571 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595491d-beb6-43ec-af16-9e5ab020a140 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:01.920290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49986cfa-6336-451d-9963-2751427aebec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:01.921163 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49986cfa-6336-451d-9963-2751427aebec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:01.921454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49986cfa-6336-451d-9963-2751427aebec admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:01.921735 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49986cfa-6336-451d-9963-2751427aebec admin admin] PATH_INFO: `/v3/projects/bea3d94bf55241dcab0f0896f77abc4c` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:01.922970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49986cfa-6336-451d-9963-2751427aebec admin admin] RBAC: Authorizing `identity:delete_project(project_id=bea3d94bf55241dcab0f0896f77abc4c)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:01.924904 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49986cfa-6336-451d-9963-2751427aebec admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:01.974247 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-49986cfa-6336-451d-9963-2751427aebec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bea3d94bf55241dcab0f0896f77abc4c', 'request_id': 'req-49986cfa-6336-451d-9963-2751427aebec'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:01.982591 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-49986cfa-6336-451d-9963-2751427aebec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bea3d94bf55241dcab0f0896f77abc4c', 'request_id': 'req-49986cfa-6336-451d-9963-2751427aebec'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:01.988368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:44:01.989604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-49986cfa-6336-451d-9963-2751427aebec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bea3d94bf55241dcab0f0896f77abc4c', 'request_id': 'req-49986cfa-6336-451d-9963-2751427aebec'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:02.006456 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-49986cfa-6336-451d-9963-2751427aebec admin admin] The token cache is being invalidate because project bea3d94bf55241dcab0f0896f77abc4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:44:02.006749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-49986cfa-6336-451d-9963-2751427aebec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:02.007421 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-49986cfa-6336-451d-9963-2751427aebec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:02.008946 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1339/2689] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:44:01 2022] DELETE /identity/v3/projects/bea3d94bf55241dcab0f0896f77abc4c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:44:02.013753 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980073aa-acc3-4cf2-ab67-212466db3dd2 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:02.014047 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980073aa-acc3-4cf2-ab67-212466db3dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:02.014333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980073aa-acc3-4cf2-ab67-212466db3dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:02.026560 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:44:02.032766 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-980073aa-acc3-4cf2-ab67-212466db3dd2 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:44:02.034492 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-980073aa-acc3-4cf2-ab67-212466db3dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:02.096006 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1351/2690] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:44:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:44:02.103812 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e6782-254e-4560-8c50-3c4655153902 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:02.104106 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e6782-254e-4560-8c50-3c4655153902 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:02.104392 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e6782-254e-4560-8c50-3c4655153902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:02.124019 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-903e6782-254e-4560-8c50-3c4655153902 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:44:02.125167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-903e6782-254e-4560-8c50-3c4655153902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:02.139341 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1340/2691] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:44:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:44:02.143556 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980073aa-acc3-4cf2-ab67-212466db3dd2 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:02.153330 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980073aa-acc3-4cf2-ab67-212466db3dd2 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:02.158187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c59db-54f9-405c-b7a6-937b7c4b17e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:02.158858 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c59db-54f9-405c-b7a6-937b7c4b17e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:02.159194 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c59db-54f9-405c-b7a6-937b7c4b17e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:02.159478 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c59db-54f9-405c-b7a6-937b7c4b17e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:02.160211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c59db-54f9-405c-b7a6-937b7c4b17e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:44:02.160527 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c59db-54f9-405c-b7a6-937b7c4b17e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:02.162112 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c59db-54f9-405c-b7a6-937b7c4b17e3 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:02.180210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1352/2692] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:44:02.185342 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e6782-254e-4560-8c50-3c4655153902 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:02.194659 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e6782-254e-4560-8c50-3c4655153902 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:02.199683 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d8c89-a333-47ec-a000-66113da4b32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:02.200466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d8c89-a333-47ec-a000-66113da4b32d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:02.201222 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d8c89-a333-47ec-a000-66113da4b32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:02.201222 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d8c89-a333-47ec-a000-66113da4b32d admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:02.201711 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d8c89-a333-47ec-a000-66113da4b32d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:02.203669 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d8c89-a333-47ec-a000-66113da4b32d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:02.229732 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1341/2693] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:44:02 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:44:02.234327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c59db-54f9-405c-b7a6-937b7c4b17e3 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:02.244298 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c59db-54f9-405c-b7a6-937b7c4b17e3 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:02.303460 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe0b98-70b3-4d14-a658-0e4bcad41e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:02.304140 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe0b98-70b3-4d14-a658-0e4bcad41e72 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:02.304431 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe0b98-70b3-4d14-a658-0e4bcad41e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:02.304716 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe0b98-70b3-4d14-a658-0e4bcad41e72 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:02.305403 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe0b98-70b3-4d14-a658-0e4bcad41e72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:02.307370 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe0b98-70b3-4d14-a658-0e4bcad41e72 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:02.328192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1353/2694] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:44:02 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:44:02.332486 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d8c89-a333-47ec-a000-66113da4b32d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:02.343187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d8c89-a333-47ec-a000-66113da4b32d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:02.347729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830ca45-f56a-4a92-aff0-d428572340d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:02.348381 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830ca45-f56a-4a92-aff0-d428572340d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:02.348666 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830ca45-f56a-4a92-aff0-d428572340d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:02.348947 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830ca45-f56a-4a92-aff0-d428572340d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:02.349684 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830ca45-f56a-4a92-aff0-d428572340d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:02.351291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830ca45-f56a-4a92-aff0-d428572340d8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:02.362179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1342/2695] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:44:02.366325 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe0b98-70b3-4d14-a658-0e4bcad41e72 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:02.376229 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe0b98-70b3-4d14-a658-0e4bcad41e72 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:02.380856 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00cc84-ada2-4ed9-814d-de6cde571373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:02.381587 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00cc84-ada2-4ed9-814d-de6cde571373 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:02.381871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00cc84-ada2-4ed9-814d-de6cde571373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:02.382153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00cc84-ada2-4ed9-814d-de6cde571373 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:02.382877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e00cc84-ada2-4ed9-814d-de6cde571373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:02.384511 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e00cc84-ada2-4ed9-814d-de6cde571373 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:02.395133 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1354/2696] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:44:02.398997 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830ca45-f56a-4a92-aff0-d428572340d8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:02.409448 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830ca45-f56a-4a92-aff0-d428572340d8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:02.414344 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92132869-09c8-4f5c-ade8-3784e17f2390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:02.415258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92132869-09c8-4f5c-ade8-3784e17f2390 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:02.415546 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92132869-09c8-4f5c-ade8-3784e17f2390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:02.415862 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92132869-09c8-4f5c-ade8-3784e17f2390 admin admin] PATH_INFO: `/v3/projects/6183038c7bc34b78bac9c779c3b9bd41/users/cba3440421214a528425fe13aad1aaa4/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:02.431287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92132869-09c8-4f5c-ade8-3784e17f2390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6183038c7bc34b78bac9c779c3b9bd41, user_id=cba3440421214a528425fe13aad1aaa4, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:02.433903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92132869-09c8-4f5c-ade8-3784e17f2390 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:02.446848 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1343/2697] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:44:02 2022] PUT /identity/v3/projects/6183038c7bc34b78bac9c779c3b9bd41/users/cba3440421214a528425fe13aad1aaa4/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:44:02.458580 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075db697-b1ce-4795-82ef-0f43212e8bee None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:02.459134 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075db697-b1ce-4795-82ef-0f43212e8bee None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:02.459485 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075db697-b1ce-4795-82ef-0f43212e8bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:02.490205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-075db697-b1ce-4795-82ef-0f43212e8bee None None] MFA Rules not processed for user `cba3440421214a528425fe13aad1aaa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:44:02.491365 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-075db697-b1ce-4795-82ef-0f43212e8bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:02.555428 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1355/2698] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:44:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:44:02.566607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92132869-09c8-4f5c-ade8-3784e17f2390 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:02.568305 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-92132869-09c8-4f5c-ade8-3784e17f2390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:02.624792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92132869-09c8-4f5c-ade8-3784e17f2390 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:02.628399 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40fe43-bea4-4ff3-8f9a-b4ce3e379258 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36de21e0b3de48dabe0f27a9db8a37aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:02.629046 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40fe43-bea4-4ff3-8f9a-b4ce3e379258 service glance] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:02.629338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40fe43-bea4-4ff3-8f9a-b4ce3e379258 service glance] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:02.629625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40fe43-bea4-4ff3-8f9a-b4ce3e379258 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:02.637799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40fe43-bea4-4ff3-8f9a-b4ce3e379258 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:02.639247 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40fe43-bea4-4ff3-8f9a-b4ce3e379258 service glance] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:02.647973 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1344/2699] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:44:02 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:44:07.415140 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075db697-b1ce-4795-82ef-0f43212e8bee None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:07.416535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-075db697-b1ce-4795-82ef-0f43212e8bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:07.422170 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:44:07.476896 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075db697-b1ce-4795-82ef-0f43212e8bee None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:07.480460 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0235afe-9084-488b-b599-47ae01f1d48d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:07.481108 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0235afe-9084-488b-b599-47ae01f1d48d service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:07.481426 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0235afe-9084-488b-b599-47ae01f1d48d service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:07.481703 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0235afe-9084-488b-b599-47ae01f1d48d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:07.483575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d0235afe-9084-488b-b599-47ae01f1d48d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:07.546135 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0235afe-9084-488b-b599-47ae01f1d48d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:07.548213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0235afe-9084-488b-b599-47ae01f1d48d service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:07.557240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1356/2700] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:44:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:44:12.644063 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=57,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:44:15.432069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1127a9-b0a5-4e9c-8e55-19c9564fe77e None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:15.432737 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1127a9-b0a5-4e9c-8e55-19c9564fe77e None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:15.432737 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1127a9-b0a5-4e9c-8e55-19c9564fe77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:15.451316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-be1127a9-b0a5-4e9c-8e55-19c9564fe77e None None] MFA Rules not processed for user `f523b529906f4fa989d0e836d6a08944`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:44:15.452514 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-be1127a9-b0a5-4e9c-8e55-19c9564fe77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:15.459546 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1345/2701] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:44:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:44:16.679693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9effe-d228-47cd-bd93-81a10862b3e4 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:16.680262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9effe-d228-47cd-bd93-81a10862b3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:16.680262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9effe-d228-47cd-bd93-81a10862b3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:16.698028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-54e9effe-d228-47cd-bd93-81a10862b3e4 None None] MFA Rules not processed for user `df1aeb2be41c49819b811ecfa47d4155`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:44:16.699327 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-54e9effe-d228-47cd-bd93-81a10862b3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:16.713645 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1127a9-b0a5-4e9c-8e55-19c9564fe77e None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:16.715105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-be1127a9-b0a5-4e9c-8e55-19c9564fe77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:16.761934 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1357/2702] 10.222.0.43 () {58 vars in 967 bytes} [Thu Jun 23 22:44:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:44:16.775336 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9effe-d228-47cd-bd93-81a10862b3e4 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:16.776718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-54e9effe-d228-47cd-bd93-81a10862b3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:16.799527 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1127a9-b0a5-4e9c-8e55-19c9564fe77e None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:16.804423 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571dcd46-530f-45ba-89f5-27a0ae0d7466 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101864afd59543e699189917391e6ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:16.805120 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dcd46-530f-45ba-89f5-27a0ae0d7466 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:16.805396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dcd46-530f-45ba-89f5-27a0ae0d7466 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:16.805675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571dcd46-530f-45ba-89f5-27a0ae0d7466 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:16.807641 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-571dcd46-530f-45ba-89f5-27a0ae0d7466 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:16.820091 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571dcd46-530f-45ba-89f5-27a0ae0d7466 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:16.821697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571dcd46-530f-45ba-89f5-27a0ae0d7466 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:16.832405 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1346/2703] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:44:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:44:16.846763 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:44:16.851754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9effe-d228-47cd-bd93-81a10862b3e4 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:16.857153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3beb08-4b96-4699-8552-880e0388c939 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ce35c43e8294f4cbc880f2219a6eb76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:16.857808 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3beb08-4b96-4699-8552-880e0388c939 service swift] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:16.858097 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3beb08-4b96-4699-8552-880e0388c939 service swift] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:16.858380 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3beb08-4b96-4699-8552-880e0388c939 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:16.866482 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3beb08-4b96-4699-8552-880e0388c939 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:16.868091 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3beb08-4b96-4699-8552-880e0388c939 service swift] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:16.876802 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1358/2704] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 22:44:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 22:44:26.828180 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=35 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:44:26.873398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=4 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:44:31.359331 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571dcd46-530f-45ba-89f5-27a0ae0d7466 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:31.422381 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571dcd46-530f-45ba-89f5-27a0ae0d7466 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:31.427465 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196a2b3-7cc7-45ca-9089-0a4dd5979c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:31.428336 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196a2b3-7cc7-45ca-9089-0a4dd5979c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:31.428627 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196a2b3-7cc7-45ca-9089-0a4dd5979c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:31.428915 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196a2b3-7cc7-45ca-9089-0a4dd5979c5b admin admin] PATH_INFO: `/v3/users/cba3440421214a528425fe13aad1aaa4` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:31.430105 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196a2b3-7cc7-45ca-9089-0a4dd5979c5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cba3440421214a528425fe13aad1aaa4)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:31.432054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196a2b3-7cc7-45ca-9089-0a4dd5979c5b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:31.477412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2196a2b3-7cc7-45ca-9089-0a4dd5979c5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cba3440421214a528425fe13aad1aaa4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:31.484252 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2196a2b3-7cc7-45ca-9089-0a4dd5979c5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cba3440421214a528425fe13aad1aaa4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:31.492094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2196a2b3-7cc7-45ca-9089-0a4dd5979c5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cba3440421214a528425fe13aad1aaa4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:31.492778 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2196a2b3-7cc7-45ca-9089-0a4dd5979c5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cba3440421214a528425fe13aad1aaa4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:31.493390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-2196a2b3-7cc7-45ca-9089-0a4dd5979c5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cba3440421214a528425fe13aad1aaa4'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:31.511229 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1347/2705] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:44:31 2022] DELETE /identity/v3/users/cba3440421214a528425fe13aad1aaa4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:44:31.515701 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20df2e3-4e84-4dc0-a0d4-efa7cc67a76e None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:31.516187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20df2e3-4e84-4dc0-a0d4-efa7cc67a76e None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:31.516293 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20df2e3-4e84-4dc0-a0d4-efa7cc67a76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:31.533255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-e20df2e3-4e84-4dc0-a0d4-efa7cc67a76e None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:44:31.534432 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e20df2e3-4e84-4dc0-a0d4-efa7cc67a76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:31.592861 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1359/2706] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:44:31.603096 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196a2b3-7cc7-45ca-9089-0a4dd5979c5b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:31.604521 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2196a2b3-7cc7-45ca-9089-0a4dd5979c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:31.663065 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196a2b3-7cc7-45ca-9089-0a4dd5979c5b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:31.666684 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf174685-d303-4691-807e-5a6c05373922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:31.667998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf174685-d303-4691-807e-5a6c05373922 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:31.668290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf174685-d303-4691-807e-5a6c05373922 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:31.668573 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf174685-d303-4691-807e-5a6c05373922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:31.677710 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf174685-d303-4691-807e-5a6c05373922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:31.679160 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf174685-d303-4691-807e-5a6c05373922 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:31.688426 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1348/2707] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:44:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:44:31.954602 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20df2e3-4e84-4dc0-a0d4-efa7cc67a76e None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:31.957278 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e20df2e3-4e84-4dc0-a0d4-efa7cc67a76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:31.969239 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20df2e3-4e84-4dc0-a0d4-efa7cc67a76e None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:31.974479 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f80e7-8c35-4507-a5ac-2a0d339657b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:31.975405 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f80e7-8c35-4507-a5ac-2a0d339657b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:31.975691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f80e7-8c35-4507-a5ac-2a0d339657b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:31.975973 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f80e7-8c35-4507-a5ac-2a0d339657b8 admin admin] PATH_INFO: `/v3/projects/6183038c7bc34b78bac9c779c3b9bd41` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:31.977263 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15f80e7-8c35-4507-a5ac-2a0d339657b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6183038c7bc34b78bac9c779c3b9bd41)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:31.979231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15f80e7-8c35-4507-a5ac-2a0d339657b8 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:32.026906 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c15f80e7-8c35-4507-a5ac-2a0d339657b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6183038c7bc34b78bac9c779c3b9bd41', 'request_id': 'req-c15f80e7-8c35-4507-a5ac-2a0d339657b8'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:32.033149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c15f80e7-8c35-4507-a5ac-2a0d339657b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6183038c7bc34b78bac9c779c3b9bd41', 'request_id': 'req-c15f80e7-8c35-4507-a5ac-2a0d339657b8'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:32.040458 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c15f80e7-8c35-4507-a5ac-2a0d339657b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6183038c7bc34b78bac9c779c3b9bd41', 'request_id': 'req-c15f80e7-8c35-4507-a5ac-2a0d339657b8'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:32.051315 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c15f80e7-8c35-4507-a5ac-2a0d339657b8 admin admin] The token cache is being invalidate because project 6183038c7bc34b78bac9c779c3b9bd41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:44:32.051614 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c15f80e7-8c35-4507-a5ac-2a0d339657b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:32.052256 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c15f80e7-8c35-4507-a5ac-2a0d339657b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:32.053838 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1360/2708] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:44:31 2022] DELETE /identity/v3/projects/6183038c7bc34b78bac9c779c3b9bd41 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:44:32.058840 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb945fec-ba1d-45c2-baa7-f4e8df310d1f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:32.059174 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb945fec-ba1d-45c2-baa7-f4e8df310d1f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:32.059476 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb945fec-ba1d-45c2-baa7-f4e8df310d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:32.078202 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cb945fec-ba1d-45c2-baa7-f4e8df310d1f None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:44:32.079443 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cb945fec-ba1d-45c2-baa7-f4e8df310d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:32.097530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:44:32.142472 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1349/2709] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:44:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:44:32.150860 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e66b9-a2b5-4f13-9d56-bc387be57407 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:32.151182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e66b9-a2b5-4f13-9d56-bc387be57407 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:32.151472 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e66b9-a2b5-4f13-9d56-bc387be57407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:32.169720 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-b91e66b9-a2b5-4f13-9d56-bc387be57407 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:44:32.170944 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b91e66b9-a2b5-4f13-9d56-bc387be57407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:32.179364 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1361/2710] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:44:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:44:32.183461 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb945fec-ba1d-45c2-baa7-f4e8df310d1f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:32.193725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb945fec-ba1d-45c2-baa7-f4e8df310d1f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:32.198327 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33e480-1821-44fb-8380-897839bc4a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:32.198914 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33e480-1821-44fb-8380-897839bc4a1c admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:32.199227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33e480-1821-44fb-8380-897839bc4a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:32.199537 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33e480-1821-44fb-8380-897839bc4a1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:32.200286 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33e480-1821-44fb-8380-897839bc4a1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:44:32.200596 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33e480-1821-44fb-8380-897839bc4a1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:32.202183 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33e480-1821-44fb-8380-897839bc4a1c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:32.214047 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1350/2711] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:44:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:44:32.218384 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e66b9-a2b5-4f13-9d56-bc387be57407 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:32.227736 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e66b9-a2b5-4f13-9d56-bc387be57407 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:32.232508 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc58f7-0621-4e89-8f40-23f49e02bec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:32.233165 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc58f7-0621-4e89-8f40-23f49e02bec2 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:32.233455 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc58f7-0621-4e89-8f40-23f49e02bec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:32.233740 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc58f7-0621-4e89-8f40-23f49e02bec2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:32.234442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc58f7-0621-4e89-8f40-23f49e02bec2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:32.236914 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc58f7-0621-4e89-8f40-23f49e02bec2 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:32.262645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1362/2712] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:44:32 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:44:32.266836 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33e480-1821-44fb-8380-897839bc4a1c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:32.276695 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33e480-1821-44fb-8380-897839bc4a1c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:32.332125 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08358381-3214-4917-8215-89bf355c287d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:32.332784 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08358381-3214-4917-8215-89bf355c287d admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:32.333071 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08358381-3214-4917-8215-89bf355c287d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:32.333353 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08358381-3214-4917-8215-89bf355c287d admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:32.334038 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08358381-3214-4917-8215-89bf355c287d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:32.335953 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08358381-3214-4917-8215-89bf355c287d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:32.355745 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1351/2713] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:44:32 2022] POST /identity/v3/users => generated 292 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:44:32.359784 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc58f7-0621-4e89-8f40-23f49e02bec2 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:32.372136 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc58f7-0621-4e89-8f40-23f49e02bec2 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:32.378038 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057df73e-c9f5-45bf-af90-3c65a094002d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:32.378706 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057df73e-c9f5-45bf-af90-3c65a094002d admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:32.379028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057df73e-c9f5-45bf-af90-3c65a094002d admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:32.379318 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057df73e-c9f5-45bf-af90-3c65a094002d admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:32.380053 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057df73e-c9f5-45bf-af90-3c65a094002d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:32.381657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057df73e-c9f5-45bf-af90-3c65a094002d admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:32.392126 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1363/2714] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:44:32.397151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08358381-3214-4917-8215-89bf355c287d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:32.407468 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08358381-3214-4917-8215-89bf355c287d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:32.413085 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a7723-021b-40a9-8809-ca71cb0663c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:32.413648 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a7723-021b-40a9-8809-ca71cb0663c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:32.413936 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a7723-021b-40a9-8809-ca71cb0663c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:32.414216 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a7723-021b-40a9-8809-ca71cb0663c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:32.415020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a7723-021b-40a9-8809-ca71cb0663c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:32.416592 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a7723-021b-40a9-8809-ca71cb0663c4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:32.427124 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1352/2715] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:44:32.431335 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057df73e-c9f5-45bf-af90-3c65a094002d admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:32.441478 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057df73e-c9f5-45bf-af90-3c65a094002d admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:32.446180 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96090d-8439-4c0d-806e-40ddeba21afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:32.447105 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96090d-8439-4c0d-806e-40ddeba21afa admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:32.447394 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96090d-8439-4c0d-806e-40ddeba21afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:32.447676 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96090d-8439-4c0d-806e-40ddeba21afa admin admin] PATH_INFO: `/v3/projects/65a0d4c968d84b02977004edf46cde48/users/f87b9f9f1adc4c2bb9da82df9d27802e/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:32.463348 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96090d-8439-4c0d-806e-40ddeba21afa admin admin] RBAC: Authorizing `identity:create_grant(project_id=65a0d4c968d84b02977004edf46cde48, user_id=f87b9f9f1adc4c2bb9da82df9d27802e, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:32.466036 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96090d-8439-4c0d-806e-40ddeba21afa admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:32.480270 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1364/2716] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:44:32 2022] PUT /identity/v3/projects/65a0d4c968d84b02977004edf46cde48/users/f87b9f9f1adc4c2bb9da82df9d27802e/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:44:32.485318 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a5028-f654-48f5-a022-ab7afacd1f88 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:32.485506 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a5028-f654-48f5-a022-ab7afacd1f88 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:32.485793 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a5028-f654-48f5-a022-ab7afacd1f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:32.504067 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-359a5028-f654-48f5-a022-ab7afacd1f88 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:44:32.505262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-359a5028-f654-48f5-a022-ab7afacd1f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:32.567796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1353/2717] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:44:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:44:32.577260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96090d-8439-4c0d-806e-40ddeba21afa admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:32.579027 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7b96090d-8439-4c0d-806e-40ddeba21afa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:32.602373 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:44:32.639805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96090d-8439-4c0d-806e-40ddeba21afa admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:32.643648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c78f33-5d79-437a-b711-b7a672b5dcd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:32.644295 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c78f33-5d79-437a-b711-b7a672b5dcd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:32.644580 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c78f33-5d79-437a-b711-b7a672b5dcd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:32.644861 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c78f33-5d79-437a-b711-b7a672b5dcd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:32.653783 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c78f33-5d79-437a-b711-b7a672b5dcd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:32.656310 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c78f33-5d79-437a-b711-b7a672b5dcd0 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:32.666435 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1365/2718] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:44:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:44:33.328156 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbacbe4-8c71-4785-a10f-9b90a89d1003 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:33.328511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbacbe4-8c71-4785-a10f-9b90a89d1003 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:33.328893 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbacbe4-8c71-4785-a10f-9b90a89d1003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:33.357381 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-0bbacbe4-8c71-4785-a10f-9b90a89d1003 None None] MFA Rules not processed for user `f87b9f9f1adc4c2bb9da82df9d27802e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:44:33.358645 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-0bbacbe4-8c71-4785-a10f-9b90a89d1003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:33.376736 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:44:33.426823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1354/2719] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:44:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:44:33.437570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c78f33-5d79-437a-b711-b7a672b5dcd0 service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:33.439180 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e5c78f33-5d79-437a-b711-b7a672b5dcd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:33.498259 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c78f33-5d79-437a-b711-b7a672b5dcd0 service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:33.501859 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a80fe-3d77-4736-b5b8-df02ae753b15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:33.503205 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a80fe-3d77-4736-b5b8-df02ae753b15 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:33.503497 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a80fe-3d77-4736-b5b8-df02ae753b15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:33.503785 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a80fe-3d77-4736-b5b8-df02ae753b15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:33.512158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a80fe-3d77-4736-b5b8-df02ae753b15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:33.513595 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a80fe-3d77-4736-b5b8-df02ae753b15 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:33.523127 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1366/2720] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:44:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:44:34.879000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c766918-7bce-4595-a6d1-ffab2cb9d4c5 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:34.879611 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c766918-7bce-4595-a6d1-ffab2cb9d4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:34.879611 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c766918-7bce-4595-a6d1-ffab2cb9d4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:34.897746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-4c766918-7bce-4595-a6d1-ffab2cb9d4c5 None None] MFA Rules not processed for user `f87b9f9f1adc4c2bb9da82df9d27802e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:44:34.899068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4c766918-7bce-4595-a6d1-ffab2cb9d4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:34.906284 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1355/2721] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:44:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:44:35.400706 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a80fe-3d77-4736-b5b8-df02ae753b15 service cinder] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:35.411022 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a80fe-3d77-4736-b5b8-df02ae753b15 service cinder] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:35.414590 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aaf4c2-d1d1-47d8-8f91-7c538d411779 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:35.415292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aaf4c2-d1d1-47d8-8f91-7c538d411779 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:35.415583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aaf4c2-d1d1-47d8-8f91-7c538d411779 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:35.415871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aaf4c2-d1d1-47d8-8f91-7c538d411779 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:35.417934 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-57aaf4c2-d1d1-47d8-8f91-7c538d411779 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:35.479422 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aaf4c2-d1d1-47d8-8f91-7c538d411779 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:35.480877 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aaf4c2-d1d1-47d8-8f91-7c538d411779 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:35.491551 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1367/2722] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:44:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:44:36.386385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c766918-7bce-4595-a6d1-ffab2cb9d4c5 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:36.387919 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-4c766918-7bce-4595-a6d1-ffab2cb9d4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:36.467847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c766918-7bce-4595-a6d1-ffab2cb9d4c5 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:36.473609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36cbd95-87be-4ae0-858a-b17027810336 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:36.474290 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36cbd95-87be-4ae0-858a-b17027810336 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:36.474569 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36cbd95-87be-4ae0-858a-b17027810336 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:36.474906 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36cbd95-87be-4ae0-858a-b17027810336 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:36.476873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-e36cbd95-87be-4ae0-858a-b17027810336 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:36.490202 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36cbd95-87be-4ae0-858a-b17027810336 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:36.492439 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36cbd95-87be-4ae0-858a-b17027810336 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:36.515137 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1356/2723] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:44:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:44:41.739589 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aaf4c2-d1d1-47d8-8f91-7c538d411779 service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:41.741087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-57aaf4c2-d1d1-47d8-8f91-7c538d411779 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:41.752042 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aaf4c2-d1d1-47d8-8f91-7c538d411779 service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:41.755849 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055638e-6406-45a8-954d-de84bc91144c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:41.756497 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055638e-6406-45a8-954d-de84bc91144c service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:41.756786 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055638e-6406-45a8-954d-de84bc91144c service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:41.757068 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055638e-6406-45a8-954d-de84bc91144c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:41.758776 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8055638e-6406-45a8-954d-de84bc91144c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:41.770602 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055638e-6406-45a8-954d-de84bc91144c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:41.772102 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055638e-6406-45a8-954d-de84bc91144c service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:41.781777 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1368/2724] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:44:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:44:44.446546 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36cbd95-87be-4ae0-858a-b17027810336 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:44.457052 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36cbd95-87be-4ae0-858a-b17027810336 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:44.460742 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898562f-cc3a-4c67-80b9-4e11d2a20fe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:44.461496 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898562f-cc3a-4c67-80b9-4e11d2a20fe2 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:44.461780 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898562f-cc3a-4c67-80b9-4e11d2a20fe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:44.462059 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898562f-cc3a-4c67-80b9-4e11d2a20fe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:44.464828 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f898562f-cc3a-4c67-80b9-4e11d2a20fe2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:44.529856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898562f-cc3a-4c67-80b9-4e11d2a20fe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:44.531642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898562f-cc3a-4c67-80b9-4e11d2a20fe2 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:44.544062 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1357/2725] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:44:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:44:49.891716 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055638e-6406-45a8-954d-de84bc91144c service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:49.893805 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8055638e-6406-45a8-954d-de84bc91144c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:49.907535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055638e-6406-45a8-954d-de84bc91144c service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:49.912381 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdcfa7b-ceb6-4ecb-9a9f-df506c842c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:49.913482 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcfa7b-ceb6-4ecb-9a9f-df506c842c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:49.913776 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcfa7b-ceb6-4ecb-9a9f-df506c842c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:49.914181 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcfa7b-ceb6-4ecb-9a9f-df506c842c69 admin admin] PATH_INFO: `/v3/users/35838f82371148dda993ca515a1be79c` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:49.915457 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdcfa7b-ceb6-4ecb-9a9f-df506c842c69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35838f82371148dda993ca515a1be79c)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:49.917451 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdcfa7b-ceb6-4ecb-9a9f-df506c842c69 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:49.972551 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:44:49.974429 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-dbdcfa7b-ceb6-4ecb-9a9f-df506c842c69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35838f82371148dda993ca515a1be79c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:49.989941 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-dbdcfa7b-ceb6-4ecb-9a9f-df506c842c69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35838f82371148dda993ca515a1be79c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.006740 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-dbdcfa7b-ceb6-4ecb-9a9f-df506c842c69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35838f82371148dda993ca515a1be79c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.014466 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-dbdcfa7b-ceb6-4ecb-9a9f-df506c842c69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35838f82371148dda993ca515a1be79c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.015316 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-dbdcfa7b-ceb6-4ecb-9a9f-df506c842c69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35838f82371148dda993ca515a1be79c'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.024608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1369/2726] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:44:49 2022] DELETE /identity/v3/users/35838f82371148dda993ca515a1be79c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:44:50.218282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898562f-cc3a-4c67-80b9-4e11d2a20fe2 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:50.229344 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898562f-cc3a-4c67-80b9-4e11d2a20fe2 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:50.261264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:44:50.292441 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f976a-2832-487d-94bd-be878de34ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:50.293568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f976a-2832-487d-94bd-be878de34ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:50.293855 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f976a-2832-487d-94bd-be878de34ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:50.294140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f976a-2832-487d-94bd-be878de34ee6 admin admin] PATH_INFO: `/v3/projects/dceb245422db46eaa1440c39b5755aca` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:50.295361 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39f976a-2832-487d-94bd-be878de34ee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dceb245422db46eaa1440c39b5755aca)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:50.297336 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39f976a-2832-487d-94bd-be878de34ee6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:50.344413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d39f976a-2832-487d-94bd-be878de34ee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dceb245422db46eaa1440c39b5755aca', 'request_id': 'req-d39f976a-2832-487d-94bd-be878de34ee6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.351199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d39f976a-2832-487d-94bd-be878de34ee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dceb245422db46eaa1440c39b5755aca', 'request_id': 'req-d39f976a-2832-487d-94bd-be878de34ee6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.357148 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d39f976a-2832-487d-94bd-be878de34ee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dceb245422db46eaa1440c39b5755aca', 'request_id': 'req-d39f976a-2832-487d-94bd-be878de34ee6'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.369350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d39f976a-2832-487d-94bd-be878de34ee6 admin admin] The token cache is being invalidate because project dceb245422db46eaa1440c39b5755aca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:44:50.369656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d39f976a-2832-487d-94bd-be878de34ee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.370337 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-d39f976a-2832-487d-94bd-be878de34ee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.371975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1358/2727] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:44:50 2022] DELETE /identity/v3/projects/dceb245422db46eaa1440c39b5755aca => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:44:50.376105 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdcfa7b-ceb6-4ecb-9a9f-df506c842c69 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:50.378397 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-dbdcfa7b-ceb6-4ecb-9a9f-df506c842c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:50.392056 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdcfa7b-ceb6-4ecb-9a9f-df506c842c69 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:50.450741 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbdf7ce-b4ec-40f9-9ec5-1133142050d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:50.451882 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdf7ce-b4ec-40f9-9ec5-1133142050d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:50.452240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdf7ce-b4ec-40f9-9ec5-1133142050d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:50.452583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdf7ce-b4ec-40f9-9ec5-1133142050d5 admin admin] PATH_INFO: `/v3/users/9ad72906339a45edbadce7983c5679ab` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:50.454108 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbdf7ce-b4ec-40f9-9ec5-1133142050d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ad72906339a45edbadce7983c5679ab)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:50.456995 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbdf7ce-b4ec-40f9-9ec5-1133142050d5 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:50.511778 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-edbdf7ce-b4ec-40f9-9ec5-1133142050d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ad72906339a45edbadce7983c5679ab'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.537888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-edbdf7ce-b4ec-40f9-9ec5-1133142050d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ad72906339a45edbadce7983c5679ab'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.549627 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-edbdf7ce-b4ec-40f9-9ec5-1133142050d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ad72906339a45edbadce7983c5679ab'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.559655 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-edbdf7ce-b4ec-40f9-9ec5-1133142050d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ad72906339a45edbadce7983c5679ab'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.560402 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-edbdf7ce-b4ec-40f9-9ec5-1133142050d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ad72906339a45edbadce7983c5679ab'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.567504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1370/2728] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:44:50 2022] DELETE /identity/v3/users/9ad72906339a45edbadce7983c5679ab => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:44:50.744530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f976a-2832-487d-94bd-be878de34ee6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:44:50.746453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d39f976a-2832-487d-94bd-be878de34ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:44:50.762720 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f976a-2832-487d-94bd-be878de34ee6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:44:50.823922 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f3db0-2b18-4424-9349-77e1bafe5882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:44:50.824756 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f3db0-2b18-4424-9349-77e1bafe5882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:44:50.825046 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f3db0-2b18-4424-9349-77e1bafe5882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:44:50.825325 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f3db0-2b18-4424-9349-77e1bafe5882 admin admin] PATH_INFO: `/v3/projects/6bb38a54c255484b8bd9f316ae754708` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:44:50.826633 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f3db0-2b18-4424-9349-77e1bafe5882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bb38a54c255484b8bd9f316ae754708)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:44:50.828557 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f3db0-2b18-4424-9349-77e1bafe5882 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:44:50.877609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-569f3db0-2b18-4424-9349-77e1bafe5882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bb38a54c255484b8bd9f316ae754708', 'request_id': 'req-569f3db0-2b18-4424-9349-77e1bafe5882'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.884963 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-569f3db0-2b18-4424-9349-77e1bafe5882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bb38a54c255484b8bd9f316ae754708', 'request_id': 'req-569f3db0-2b18-4424-9349-77e1bafe5882'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.889770 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:44:50.891008 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-569f3db0-2b18-4424-9349-77e1bafe5882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bb38a54c255484b8bd9f316ae754708', 'request_id': 'req-569f3db0-2b18-4424-9349-77e1bafe5882'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.904664 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-569f3db0-2b18-4424-9349-77e1bafe5882 admin admin] The token cache is being invalidate because project 6bb38a54c255484b8bd9f316ae754708 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:44:50.904957 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-569f3db0-2b18-4424-9349-77e1bafe5882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.905620 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-569f3db0-2b18-4424-9349-77e1bafe5882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:44:50.907211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1359/2729] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:44:50 2022] DELETE /identity/v3/projects/6bb38a54c255484b8bd9f316ae754708 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:45:00.562776 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=66,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:00.903966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:07.115771 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af352fe1-a123-4c83-b9b1-c289fe589992 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:07.116306 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af352fe1-a123-4c83-b9b1-c289fe589992 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:07.116407 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af352fe1-a123-4c83-b9b1-c289fe589992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:07.137069 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-af352fe1-a123-4c83-b9b1-c289fe589992 None None] MFA Rules not processed for user `1a93ea9f0a3f47d68e53c464c1d52c92`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:07.138318 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-af352fe1-a123-4c83-b9b1-c289fe589992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:07.194983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1371/2730] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:45:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:17.187314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=25 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:45.637821 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788ed67-3bce-4446-bb4d-ded8639da908 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:45.638390 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788ed67-3bce-4446-bb4d-ded8639da908 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:45.638500 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788ed67-3bce-4446-bb4d-ded8639da908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:45.659925 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1788ed67-3bce-4446-bb4d-ded8639da908 None None] MFA Rules not processed for user `f87b9f9f1adc4c2bb9da82df9d27802e`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:45:45.661127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1788ed67-3bce-4446-bb4d-ded8639da908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:45.716594 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1360/2731] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:45:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:45:55.710049 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=25 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:45:57.012507 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af352fe1-a123-4c83-b9b1-c289fe589992 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:57.013862 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-af352fe1-a123-4c83-b9b1-c289fe589992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:57.073259 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af352fe1-a123-4c83-b9b1-c289fe589992 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:57.076793 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf740fc-ee09-4cdf-8d75-47f8de0fafcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:57.077461 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf740fc-ee09-4cdf-8d75-47f8de0fafcc service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:57.077747 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf740fc-ee09-4cdf-8d75-47f8de0fafcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:57.078026 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf740fc-ee09-4cdf-8d75-47f8de0fafcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:57.079794 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-6cf740fc-ee09-4cdf-8d75-47f8de0fafcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:57.157742 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf740fc-ee09-4cdf-8d75-47f8de0fafcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:57.159240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf740fc-ee09-4cdf-8d75-47f8de0fafcc service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:57.172210 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1372/2732] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:45:58.347682 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788ed67-3bce-4446-bb4d-ded8639da908 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:45:58.357937 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788ed67-3bce-4446-bb4d-ded8639da908 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:45:58.361396 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcd57d-5327-463b-8b0b-f171c0550240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:45:58.362040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcd57d-5327-463b-8b0b-f171c0550240 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:45:58.362324 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcd57d-5327-463b-8b0b-f171c0550240 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:45:58.362609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcd57d-5327-463b-8b0b-f171c0550240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:45:58.364413 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-49dcd57d-5327-463b-8b0b-f171c0550240 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:45:58.377606 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcd57d-5327-463b-8b0b-f171c0550240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:45:58.378980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcd57d-5327-463b-8b0b-f171c0550240 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:45:58.389688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1361/2733] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:45:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:07.167104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=54 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:08.384942 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=6 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:28.573926 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26a3d9-37a2-4b55-b9d2-c672aa1c22a8 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:28.574412 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26a3d9-37a2-4b55-b9d2-c672aa1c22a8 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:28.574529 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26a3d9-37a2-4b55-b9d2-c672aa1c22a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:28.592192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-1e26a3d9-37a2-4b55-b9d2-c672aa1c22a8 None None] MFA Rules not processed for user `df1aeb2be41c49819b811ecfa47d4155`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:28.593379 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1e26a3d9-37a2-4b55-b9d2-c672aa1c22a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:28.652462 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1373/2734] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 22:46:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:28.702014 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcd57d-5327-463b-8b0b-f171c0550240 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:28.703524 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-49dcd57d-5327-463b-8b0b-f171c0550240 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:28.775472 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcd57d-5327-463b-8b0b-f171c0550240 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:28.781287 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408b2d9-1fb6-4223-a7cf-75528b493ec2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ce35c43e8294f4cbc880f2219a6eb76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:28.782022 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408b2d9-1fb6-4223-a7cf-75528b493ec2 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:28.782322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408b2d9-1fb6-4223-a7cf-75528b493ec2 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:28.782628 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408b2d9-1fb6-4223-a7cf-75528b493ec2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:28.792276 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408b2d9-1fb6-4223-a7cf-75528b493ec2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:28.794116 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408b2d9-1fb6-4223-a7cf-75528b493ec2 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:28.803255 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1362/2735] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 22:46:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:46:36.950860 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26a3d9-37a2-4b55-b9d2-c672aa1c22a8 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:36.952496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1e26a3d9-37a2-4b55-b9d2-c672aa1c22a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:36.987360 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26a3d9-37a2-4b55-b9d2-c672aa1c22a8 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:37.005686 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd7013-5a59-4b12-8da2-606acd6d0fb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:37.007780 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd7013-5a59-4b12-8da2-606acd6d0fb6 service nova] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:37.008055 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd7013-5a59-4b12-8da2-606acd6d0fb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:37.008309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd7013-5a59-4b12-8da2-606acd6d0fb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:37.061188 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-9efd7013-5a59-4b12-8da2-606acd6d0fb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:37.098971 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd7013-5a59-4b12-8da2-606acd6d0fb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:37.100717 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd7013-5a59-4b12-8da2-606acd6d0fb6 service nova] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:37.120252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1374/2736] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:38.799197 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=32 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:44.772430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3aecda-f573-4493-91cc-288a76849786 None None] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:44.773095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3aecda-f573-4493-91cc-288a76849786 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:44.773095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3aecda-f573-4493-91cc-288a76849786 None None] PATH_INFO: `/` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:44.774660 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1363/2737] 10.222.0.43 () {54 vars in 921 bytes} [Thu Jun 23 22:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 22:46:44.779781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67193a20-9b69-49f2-ac8e-991801a65331 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:44.780277 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67193a20-9b69-49f2-ac8e-991801a65331 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:44.780389 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67193a20-9b69-49f2-ac8e-991801a65331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:44.802487 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-67193a20-9b69-49f2-ac8e-991801a65331 None None] MFA Rules not processed for user `35d686923ef541819de3c77847d38511`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:44.803719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-67193a20-9b69-49f2-ac8e-991801a65331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:44.813993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1375/2738] 10.222.0.43 () {58 vars in 1018 bytes} [Thu Jun 23 22:46:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:44.828111 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3aecda-f573-4493-91cc-288a76849786 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:44.830160 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-ca3aecda-f573-4493-91cc-288a76849786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:44.898436 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3aecda-f573-4493-91cc-288a76849786 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:44.902090 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afb6f2-f1e1-4364-b34b-e7a7a8fb5441 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36de21e0b3de48dabe0f27a9db8a37aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:44.902662 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afb6f2-f1e1-4364-b34b-e7a7a8fb5441 service glance] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:44.902977 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afb6f2-f1e1-4364-b34b-e7a7a8fb5441 service glance] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:44.903264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afb6f2-f1e1-4364-b34b-e7a7a8fb5441 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:44.914852 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afb6f2-f1e1-4364-b34b-e7a7a8fb5441 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:44.916795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afb6f2-f1e1-4364-b34b-e7a7a8fb5441 service glance] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:44.928997 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1364/2739] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:46:45.116898 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67193a20-9b69-49f2-ac8e-991801a65331 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:45.118269 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-67193a20-9b69-49f2-ac8e-991801a65331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:45.188612 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67193a20-9b69-49f2-ac8e-991801a65331 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:45.194252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85eb75-d919-42f8-9f1f-2255f2d21c34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101864afd59543e699189917391e6ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:45.194900 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85eb75-d919-42f8-9f1f-2255f2d21c34 service placement] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:45.195220 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85eb75-d919-42f8-9f1f-2255f2d21c34 service placement] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:45.195503 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85eb75-d919-42f8-9f1f-2255f2d21c34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:45.197260 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-0f85eb75-d919-42f8-9f1f-2255f2d21c34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:45.209180 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85eb75-d919-42f8-9f1f-2255f2d21c34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:45.210761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85eb75-d919-42f8-9f1f-2255f2d21c34 service placement] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:45.220577 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1376/2740] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:45.948514 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afb6f2-f1e1-4364-b34b-e7a7a8fb5441 service glance] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:45.958381 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afb6f2-f1e1-4364-b34b-e7a7a8fb5441 service glance] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:45.962192 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558baf4-078f-486a-9f50-5af9dd5d099f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:45.962771 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558baf4-078f-486a-9f50-5af9dd5d099f service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:45.963073 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558baf4-078f-486a-9f50-5af9dd5d099f service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:45.963363 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558baf4-078f-486a-9f50-5af9dd5d099f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:45.965335 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-3558baf4-078f-486a-9f50-5af9dd5d099f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:45.976570 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558baf4-078f-486a-9f50-5af9dd5d099f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:45.977999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558baf4-078f-486a-9f50-5af9dd5d099f service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:45.988334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1365/2741] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:46.784116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5deced-a717-4ad9-bc9a-cb107db94af3 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:46.784698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5deced-a717-4ad9-bc9a-cb107db94af3 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:46.784817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5deced-a717-4ad9-bc9a-cb107db94af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:46.815226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-5a5deced-a717-4ad9-bc9a-cb107db94af3 None None] MFA Rules not processed for user `df1aeb2be41c49819b811ecfa47d4155`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:46.815226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-5a5deced-a717-4ad9-bc9a-cb107db94af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:46.823158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1377/2742] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 22:46:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:46:46.836172 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558baf4-078f-486a-9f50-5af9dd5d099f service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:46.848663 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558baf4-078f-486a-9f50-5af9dd5d099f service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:46.855573 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e7def-286e-41b6-905f-79ad68478ebc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ce35c43e8294f4cbc880f2219a6eb76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:46.856459 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e7def-286e-41b6-905f-79ad68478ebc service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:46.856749 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e7def-286e-41b6-905f-79ad68478ebc service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:46.857041 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7e7def-286e-41b6-905f-79ad68478ebc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:46.866637 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e7def-286e-41b6-905f-79ad68478ebc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:46.868593 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7e7def-286e-41b6-905f-79ad68478ebc service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:46.877450 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1366/2743] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 22:46:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:46:47.387721 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5deced-a717-4ad9-bc9a-cb107db94af3 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:47.389399 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-5a5deced-a717-4ad9-bc9a-cb107db94af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:47.526449 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5deced-a717-4ad9-bc9a-cb107db94af3 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:47.537721 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299f539-8017-4147-9caf-3c852e472a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:47.539309 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299f539-8017-4147-9caf-3c852e472a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:47.540004 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299f539-8017-4147-9caf-3c852e472a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:47.540674 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299f539-8017-4147-9caf-3c852e472a91 admin admin] PATH_INFO: `/v3/users/f87b9f9f1adc4c2bb9da82df9d27802e` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:47.542821 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299f539-8017-4147-9caf-3c852e472a91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f87b9f9f1adc4c2bb9da82df9d27802e)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:47.547409 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299f539-8017-4147-9caf-3c852e472a91 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:47.563736 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=100 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:47.629249 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b299f539-8017-4147-9caf-3c852e472a91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f87b9f9f1adc4c2bb9da82df9d27802e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:47.650834 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b299f539-8017-4147-9caf-3c852e472a91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f87b9f9f1adc4c2bb9da82df9d27802e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:47.661030 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b299f539-8017-4147-9caf-3c852e472a91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f87b9f9f1adc4c2bb9da82df9d27802e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:47.676247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b299f539-8017-4147-9caf-3c852e472a91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f87b9f9f1adc4c2bb9da82df9d27802e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:47.677466 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b299f539-8017-4147-9caf-3c852e472a91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f87b9f9f1adc4c2bb9da82df9d27802e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:47.701182 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1378/2744] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:46:47 2022] DELETE /identity/v3/users/f87b9f9f1adc4c2bb9da82df9d27802e => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:47.969864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e7def-286e-41b6-905f-79ad68478ebc service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:47.992225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7e7def-286e-41b6-905f-79ad68478ebc service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:48.161364 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcce1f3-eeda-4619-bbc3-e6968fdedaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:48.167185 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcce1f3-eeda-4619-bbc3-e6968fdedaea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:48.170991 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcce1f3-eeda-4619-bbc3-e6968fdedaea admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:48.171385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcce1f3-eeda-4619-bbc3-e6968fdedaea admin admin] PATH_INFO: `/v3/projects/65a0d4c968d84b02977004edf46cde48` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:48.177950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcce1f3-eeda-4619-bbc3-e6968fdedaea admin admin] RBAC: Authorizing `identity:delete_project(project_id=65a0d4c968d84b02977004edf46cde48)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:48.180083 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcce1f3-eeda-4619-bbc3-e6968fdedaea admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:48.247487 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ddcce1f3-eeda-4619-bbc3-e6968fdedaea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65a0d4c968d84b02977004edf46cde48', 'request_id': 'req-ddcce1f3-eeda-4619-bbc3-e6968fdedaea'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:48.256482 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ddcce1f3-eeda-4619-bbc3-e6968fdedaea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65a0d4c968d84b02977004edf46cde48', 'request_id': 'req-ddcce1f3-eeda-4619-bbc3-e6968fdedaea'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:48.263625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ddcce1f3-eeda-4619-bbc3-e6968fdedaea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65a0d4c968d84b02977004edf46cde48', 'request_id': 'req-ddcce1f3-eeda-4619-bbc3-e6968fdedaea'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:48.279283 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ddcce1f3-eeda-4619-bbc3-e6968fdedaea admin admin] The token cache is being invalidate because project 65a0d4c968d84b02977004edf46cde48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:46:48.281561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ddcce1f3-eeda-4619-bbc3-e6968fdedaea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:48.282368 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-ddcce1f3-eeda-4619-bbc3-e6968fdedaea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:46:48.288871 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1367/2745] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:46:47 2022] DELETE /identity/v3/projects/65a0d4c968d84b02977004edf46cde48 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:48.291811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69490118-d748-4f79-b5d0-ee2a631a8fef None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:48.292702 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69490118-d748-4f79-b5d0-ee2a631a8fef None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:48.293140 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69490118-d748-4f79-b5d0-ee2a631a8fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:48.318189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-69490118-d748-4f79-b5d0-ee2a631a8fef None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:48.318189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-69490118-d748-4f79-b5d0-ee2a631a8fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:48.464456 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1379/2746] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:48.476463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb41c99-0344-4f08-8f61-fd8784ead704 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:48.477041 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb41c99-0344-4f08-8f61-fd8784ead704 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:48.477492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb41c99-0344-4f08-8f61-fd8784ead704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:48.507950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:48.527622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-bbb41c99-0344-4f08-8f61-fd8784ead704 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:48.529173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-bbb41c99-0344-4f08-8f61-fd8784ead704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:48.539013 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1368/2747] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:48.543251 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69490118-d748-4f79-b5d0-ee2a631a8fef None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:48.564811 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69490118-d748-4f79-b5d0-ee2a631a8fef None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:48.581018 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91c7e7-cd40-4f4c-bf6c-f3b4b38cbb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:48.582329 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91c7e7-cd40-4f4c-bf6c-f3b4b38cbb2b admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:48.582623 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91c7e7-cd40-4f4c-bf6c-f3b4b38cbb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:48.582909 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91c7e7-cd40-4f4c-bf6c-f3b4b38cbb2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:48.584958 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91c7e7-cd40-4f4c-bf6c-f3b4b38cbb2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:46:48.585279 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91c7e7-cd40-4f4c-bf6c-f3b4b38cbb2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:48.586974 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91c7e7-cd40-4f4c-bf6c-f3b4b38cbb2b admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:48.608468 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1380/2748] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:46:48.613873 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb41c99-0344-4f08-8f61-fd8784ead704 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:48.637623 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb41c99-0344-4f08-8f61-fd8784ead704 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:48.653682 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c7329-60d2-4a61-a55b-52ee1ed5879b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:48.654721 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c7329-60d2-4a61-a55b-52ee1ed5879b admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:48.655231 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c7329-60d2-4a61-a55b-52ee1ed5879b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:48.655675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c7329-60d2-4a61-a55b-52ee1ed5879b admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:48.656565 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c7329-60d2-4a61-a55b-52ee1ed5879b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:48.665450 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c7329-60d2-4a61-a55b-52ee1ed5879b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:48.724416 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1369/2749] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:48 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:48.729636 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91c7e7-cd40-4f4c-bf6c-f3b4b38cbb2b admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:48.745994 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91c7e7-cd40-4f4c-bf6c-f3b4b38cbb2b admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:48.857137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf7ec3-5a65-451f-a1c6-d28d83f62eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:48.858993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf7ec3-5a65-451f-a1c6-d28d83f62eff admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:48.858993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf7ec3-5a65-451f-a1c6-d28d83f62eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:48.858993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf7ec3-5a65-451f-a1c6-d28d83f62eff admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:48.859355 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf7ec3-5a65-451f-a1c6-d28d83f62eff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:48.862149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf7ec3-5a65-451f-a1c6-d28d83f62eff admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:48.890087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c7329-60d2-4a61-a55b-52ee1ed5879b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:48.894896 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1381/2750] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:48 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:46:48.906897 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c7329-60d2-4a61-a55b-52ee1ed5879b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:48.913265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1dc33-4bf4-4136-949b-875cba9ca99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:48.921127 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1dc33-4bf4-4136-949b-875cba9ca99d admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:48.922403 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1dc33-4bf4-4136-949b-875cba9ca99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:48.922856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1dc33-4bf4-4136-949b-875cba9ca99d admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:48.923898 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea1dc33-4bf4-4136-949b-875cba9ca99d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:48.927144 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea1dc33-4bf4-4136-949b-875cba9ca99d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:48.946980 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1370/2751] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:46:48.953978 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf7ec3-5a65-451f-a1c6-d28d83f62eff admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:48.967692 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:48.979678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf7ec3-5a65-451f-a1c6-d28d83f62eff admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:48.993115 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666d925-403a-491e-b308-63188b738647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:48.994642 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666d925-403a-491e-b308-63188b738647 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:48.995731 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666d925-403a-491e-b308-63188b738647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:48.996496 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666d925-403a-491e-b308-63188b738647 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:48.999238 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666d925-403a-491e-b308-63188b738647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:49.001500 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666d925-403a-491e-b308-63188b738647 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:49.023566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1382/2752] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:46:49.028767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1dc33-4bf4-4136-949b-875cba9ca99d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:49.050454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea1dc33-4bf4-4136-949b-875cba9ca99d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:49.062084 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96015878-8718-4ccd-a675-546b1cc35a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:49.066517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96015878-8718-4ccd-a675-546b1cc35a90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:49.067090 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96015878-8718-4ccd-a675-546b1cc35a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:49.067561 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96015878-8718-4ccd-a675-546b1cc35a90 admin admin] PATH_INFO: `/v3/projects/cea45368b61d4748a164cc6b76293749/users/6867c020bf2a4db3b0a4cebb32c64aa3/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:49.106053 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96015878-8718-4ccd-a675-546b1cc35a90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea45368b61d4748a164cc6b76293749, user_id=6867c020bf2a4db3b0a4cebb32c64aa3, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:49.109222 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96015878-8718-4ccd-a675-546b1cc35a90 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:49.132856 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1371/2753] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:49 2022] PUT /identity/v3/projects/cea45368b61d4748a164cc6b76293749/users/6867c020bf2a4db3b0a4cebb32c64aa3/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:46:49.137711 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73850-2677-4c3f-8243-490c3ca96389 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:49.138039 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73850-2677-4c3f-8243-490c3ca96389 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:49.138430 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73850-2677-4c3f-8243-490c3ca96389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:49.163579 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-f5a73850-2677-4c3f-8243-490c3ca96389 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:49.164965 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-f5a73850-2677-4c3f-8243-490c3ca96389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:49.299935 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1383/2754] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:46:49.310329 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96015878-8718-4ccd-a675-546b1cc35a90 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:49.312432 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-96015878-8718-4ccd-a675-546b1cc35a90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:49.414988 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96015878-8718-4ccd-a675-546b1cc35a90 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:49.428864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4e16e-f5d4-4575-a907-bbcf9a00e6de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:49.429630 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4e16e-f5d4-4575-a907-bbcf9a00e6de service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:49.429992 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4e16e-f5d4-4575-a907-bbcf9a00e6de service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:49.431154 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4e16e-f5d4-4575-a907-bbcf9a00e6de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:49.443937 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4e16e-f5d4-4575-a907-bbcf9a00e6de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:49.445401 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4e16e-f5d4-4575-a907-bbcf9a00e6de service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:49.458818 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1372/2755] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:46:55.414936 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938a057-aedb-4f45-8494-03359bff39d0 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.415531 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938a057-aedb-4f45-8494-03359bff39d0 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.415531 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938a057-aedb-4f45-8494-03359bff39d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.446072 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-4938a057-aedb-4f45-8494-03359bff39d0 None None] MFA Rules not processed for user `6867c020bf2a4db3b0a4cebb32c64aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:46:55.447358 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-4938a057-aedb-4f45-8494-03359bff39d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:55.465976 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4e16e-f5d4-4575-a907-bbcf9a00e6de service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:55.467536 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b4a4e16e-f5d4-4575-a907-bbcf9a00e6de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:55.535936 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1384/2756] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:46:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:46:55.540141 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938a057-aedb-4f45-8494-03359bff39d0 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:55.553352 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938a057-aedb-4f45-8494-03359bff39d0 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:55.557298 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4e16e-f5d4-4575-a907-bbcf9a00e6de service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:55.558359 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d3750-b39e-49d1-8284-6dd17a1f5e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:55.559153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d3750-b39e-49d1-8284-6dd17a1f5e98 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.559608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d3750-b39e-49d1-8284-6dd17a1f5e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.559845 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d3750-b39e-49d1-8284-6dd17a1f5e98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.560604 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d3750-b39e-49d1-8284-6dd17a1f5e98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:55.562528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d3750-b39e-49d1-8284-6dd17a1f5e98 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:55.563048 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1635e-2b35-40c3-b4d3-0d12d9aa0736 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:55.563725 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1635e-2b35-40c3-b4d3-0d12d9aa0736 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.563996 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1635e-2b35-40c3-b4d3-0d12d9aa0736 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.564282 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1635e-2b35-40c3-b4d3-0d12d9aa0736 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.566452 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b8d1635e-2b35-40c3-b4d3-0d12d9aa0736 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:46:55.595213 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1385/2757] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:46:55 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:46:55.599775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d3750-b39e-49d1-8284-6dd17a1f5e98 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:55.624353 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:55.644423 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1635e-2b35-40c3-b4d3-0d12d9aa0736 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:55.646042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1635e-2b35-40c3-b4d3-0d12d9aa0736 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:55.655724 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1373/2758] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:46:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:46:55.666946 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d3750-b39e-49d1-8284-6dd17a1f5e98 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:55.672507 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9961e83-d47a-449a-9f2e-896dff1f6c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:55.673271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9961e83-d47a-449a-9f2e-896dff1f6c51 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.673567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9961e83-d47a-449a-9f2e-896dff1f6c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.673956 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9961e83-d47a-449a-9f2e-896dff1f6c51 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.674734 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9961e83-d47a-449a-9f2e-896dff1f6c51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:55.677525 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9961e83-d47a-449a-9f2e-896dff1f6c51 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:55.686086 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:46:55.711937 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1386/2759] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:46:55 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:46:55.715995 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1635e-2b35-40c3-b4d3-0d12d9aa0736 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:55.732502 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1635e-2b35-40c3-b4d3-0d12d9aa0736 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:55.744317 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ab176-57f8-4c8d-9d22-0f3a923ef135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:55.745028 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ab176-57f8-4c8d-9d22-0f3a923ef135 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.745317 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ab176-57f8-4c8d-9d22-0f3a923ef135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.745667 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ab176-57f8-4c8d-9d22-0f3a923ef135 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.746832 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ab176-57f8-4c8d-9d22-0f3a923ef135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:55.748494 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849ab176-57f8-4c8d-9d22-0f3a923ef135 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:55.760213 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1374/2760] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:55.764416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9961e83-d47a-449a-9f2e-896dff1f6c51 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:55.779331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9961e83-d47a-449a-9f2e-896dff1f6c51 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:55.792292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6e0ae-ef25-47f9-8313-efdbadd514ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:55.798266 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6e0ae-ef25-47f9-8313-efdbadd514ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.798584 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6e0ae-ef25-47f9-8313-efdbadd514ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.798896 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6e0ae-ef25-47f9-8313-efdbadd514ca admin admin] PATH_INFO: `/v3/projects/19cfa8f118ab4adf804fa31768a7e8cd/users/afbd6e91815a4b4db97ecb90ae9dfef3/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.819046 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6e0ae-ef25-47f9-8313-efdbadd514ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=19cfa8f118ab4adf804fa31768a7e8cd, user_id=afbd6e91815a4b4db97ecb90ae9dfef3, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:55.821927 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6e0ae-ef25-47f9-8313-efdbadd514ca admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:55.838954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1387/2761] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:55 2022] PUT /identity/v3/projects/19cfa8f118ab4adf804fa31768a7e8cd/users/afbd6e91815a4b4db97ecb90ae9dfef3/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:46:55.843010 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ab176-57f8-4c8d-9d22-0f3a923ef135 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:55.858017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ab176-57f8-4c8d-9d22-0f3a923ef135 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:55.946338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6798c11-5171-47c8-8848-47ac124aa23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:55.946338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6798c11-5171-47c8-8848-47ac124aa23b admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.946338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6798c11-5171-47c8-8848-47ac124aa23b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.946338 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6798c11-5171-47c8-8848-47ac124aa23b admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.946737 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6798c11-5171-47c8-8848-47ac124aa23b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:55.946737 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6798c11-5171-47c8-8848-47ac124aa23b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:55.956870 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1375/2762] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:46:55.960957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6e0ae-ef25-47f9-8313-efdbadd514ca admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:46:55.973165 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6e0ae-ef25-47f9-8313-efdbadd514ca admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:46:55.978136 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947cd3e-d943-4aca-9484-e04a0964c9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:46:55.979092 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947cd3e-d943-4aca-9484-e04a0964c9e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:46:55.979383 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947cd3e-d943-4aca-9484-e04a0964c9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:46:55.979670 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947cd3e-d943-4aca-9484-e04a0964c9e7 admin admin] PATH_INFO: `/v3/projects/19cfa8f118ab4adf804fa31768a7e8cd/users/afbd6e91815a4b4db97ecb90ae9dfef3/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:46:55.983781 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947cd3e-d943-4aca-9484-e04a0964c9e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19cfa8f118ab4adf804fa31768a7e8cd, user_id=afbd6e91815a4b4db97ecb90ae9dfef3, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:46:55.986797 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947cd3e-d943-4aca-9484-e04a0964c9e7 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:46:55.999945 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1388/2763] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:46:55 2022] PUT /identity/v3/projects/19cfa8f118ab4adf804fa31768a7e8cd/users/afbd6e91815a4b4db97ecb90ae9dfef3/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:00.819079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6798c11-5171-47c8-8848-47ac124aa23b admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:00.978044 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6798c11-5171-47c8-8848-47ac124aa23b admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:01.088457 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:01.097847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36455f-d154-4d69-b1b2-54338149b0d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d686923ef541819de3c77847d38511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'ResellerAdmin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:01.098444 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36455f-d154-4d69-b1b2-54338149b0d0 service nova] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:01.098795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36455f-d154-4d69-b1b2-54338149b0d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:01.099473 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36455f-d154-4d69-b1b2-54338149b0d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:01.101847 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-be36455f-d154-4d69-b1b2-54338149b0d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:01.117759 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36455f-d154-4d69-b1b2-54338149b0d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:01.120142 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36455f-d154-4d69-b1b2-54338149b0d0 service nova] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:01.141764 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1376/2764] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:01.798885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fcd7c-f4e8-4a18-b27c-2ac5685ac4b2 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:01.799443 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fcd7c-f4e8-4a18-b27c-2ac5685ac4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:01.801722 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fcd7c-f4e8-4a18-b27c-2ac5685ac4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:01.830375 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-cc7fcd7c-f4e8-4a18-b27c-2ac5685ac4b2 None None] MFA Rules not processed for user `afbd6e91815a4b4db97ecb90ae9dfef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:01.832326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-cc7fcd7c-f4e8-4a18-b27c-2ac5685ac4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:01.931490 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36455f-d154-4d69-b1b2-54338149b0d0 service nova] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:01.936863 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1389/2765] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:47:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:01.950261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fcd7c-f4e8-4a18-b27c-2ac5685ac4b2 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:01.951772 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-cc7fcd7c-f4e8-4a18-b27c-2ac5685ac4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:02.010970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36455f-d154-4d69-b1b2-54338149b0d0 service nova] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:02.015042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc43b3-f70c-4c7a-86be-22030562423b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:02.016054 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc43b3-f70c-4c7a-86be-22030562423b service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:02.016584 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc43b3-f70c-4c7a-86be-22030562423b service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:02.016955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc43b3-f70c-4c7a-86be-22030562423b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:02.018906 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fcd7c-f4e8-4a18-b27c-2ac5685ac4b2 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:02.019097 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f8bc43b3-f70c-4c7a-86be-22030562423b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:02.022480 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38451d6-fece-4af2-9de2-03a18e5ddd70 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36de21e0b3de48dabe0f27a9db8a37aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:02.023229 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38451d6-fece-4af2-9de2-03a18e5ddd70 service glance] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:02.023518 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38451d6-fece-4af2-9de2-03a18e5ddd70 service glance] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:02.023800 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38451d6-fece-4af2-9de2-03a18e5ddd70 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:02.087633 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc43b3-f70c-4c7a-86be-22030562423b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:02.088314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:02.089198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc43b3-f70c-4c7a-86be-22030562423b service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:02.092223 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38451d6-fece-4af2-9de2-03a18e5ddd70 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:02.093949 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38451d6-fece-4af2-9de2-03a18e5ddd70 service glance] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:02.100853 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1377/2766] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:47:02.105290 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1390/2767] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:47:02.254461 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744993f-cff8-4b51-87f4-f7ab93debd02 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:02.254752 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744993f-cff8-4b51-87f4-f7ab93debd02 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:02.255084 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744993f-cff8-4b51-87f4-f7ab93debd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:02.286540 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7744993f-cff8-4b51-87f4-f7ab93debd02 None None] MFA Rules not processed for user `df1aeb2be41c49819b811ecfa47d4155`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:02.290004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7744993f-cff8-4b51-87f4-f7ab93debd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:02.375122 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1378/2768] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 22:47:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:02.383838 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38451d6-fece-4af2-9de2-03a18e5ddd70 service glance] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:02.386470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-a38451d6-fece-4af2-9de2-03a18e5ddd70 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:02.475442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38451d6-fece-4af2-9de2-03a18e5ddd70 service glance] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:02.482154 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51ef1f-17dc-4841-b7e2-8d1093f3945f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ce35c43e8294f4cbc880f2219a6eb76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:02.482823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51ef1f-17dc-4841-b7e2-8d1093f3945f service swift] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:02.483140 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51ef1f-17dc-4841-b7e2-8d1093f3945f service swift] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:02.483419 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51ef1f-17dc-4841-b7e2-8d1093f3945f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:02.493259 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51ef1f-17dc-4841-b7e2-8d1093f3945f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:02.495144 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51ef1f-17dc-4841-b7e2-8d1093f3945f service swift] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:02.505426 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1391/2769] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 22:47:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:47:03.070585 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de956b-bfc0-4435-bf7b-03d2fbdc0a31 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:03.071827 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de956b-bfc0-4435-bf7b-03d2fbdc0a31 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:03.072288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de956b-bfc0-4435-bf7b-03d2fbdc0a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:03.104558 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-15de956b-bfc0-4435-bf7b-03d2fbdc0a31 None None] MFA Rules not processed for user `df1aeb2be41c49819b811ecfa47d4155`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:03.106189 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-15de956b-bfc0-4435-bf7b-03d2fbdc0a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:03.117468 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1379/2770] 10.222.0.43 () {58 vars in 967 bytes} [Thu Jun 23 22:47:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:47:03.132187 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51ef1f-17dc-4841-b7e2-8d1093f3945f service swift] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:03.150211 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51ef1f-17dc-4841-b7e2-8d1093f3945f service swift] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:03.164002 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ab3f0-6ab0-40ff-aacc-ba3bb4bd8985 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ce35c43e8294f4cbc880f2219a6eb76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:03.165465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ab3f0-6ab0-40ff-aacc-ba3bb4bd8985 service swift] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:03.165465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ab3f0-6ab0-40ff-aacc-ba3bb4bd8985 service swift] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:03.165465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ab3f0-6ab0-40ff-aacc-ba3bb4bd8985 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:03.175901 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ab3f0-6ab0-40ff-aacc-ba3bb4bd8985 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:03.178002 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ab3f0-6ab0-40ff-aacc-ba3bb4bd8985 service swift] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:03.192867 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1392/2771] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 22:47:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 22:47:06.774218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de956b-bfc0-4435-bf7b-03d2fbdc0a31 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:06.774218 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-15de956b-bfc0-4435-bf7b-03d2fbdc0a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:07.148993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=100 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:07.185181 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de956b-bfc0-4435-bf7b-03d2fbdc0a31 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:07.191145 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6e5d7-9dc3-4a7c-9aea-48ed95bbce93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101864afd59543e699189917391e6ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:07.191824 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6e5d7-9dc3-4a7c-9aea-48ed95bbce93 service placement] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:07.192114 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6e5d7-9dc3-4a7c-9aea-48ed95bbce93 service placement] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:07.192398 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6e5d7-9dc3-4a7c-9aea-48ed95bbce93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:07.194262 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-68b6e5d7-9dc3-4a7c-9aea-48ed95bbce93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:07.208833 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6e5d7-9dc3-4a7c-9aea-48ed95bbce93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:07.210448 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b6e5d7-9dc3-4a7c-9aea-48ed95bbce93 service placement] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:07.228153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1380/2772] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:11.481469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ab3f0-6ab0-40ff-aacc-ba3bb4bd8985 service swift] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:11.483046 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-7b1ab3f0-6ab0-40ff-aacc-ba3bb4bd8985 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:11.499926 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ab3f0-6ab0-40ff-aacc-ba3bb4bd8985 service swift] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:11.507434 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2767798-7d00-415a-a9ef-edfa8b6cef10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:11.508601 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2767798-7d00-415a-a9ef-edfa8b6cef10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:11.509078 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2767798-7d00-415a-a9ef-edfa8b6cef10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:11.509535 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2767798-7d00-415a-a9ef-edfa8b6cef10 admin admin] PATH_INFO: `/v3/users/f523b529906f4fa989d0e836d6a08944` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:11.510775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2767798-7d00-415a-a9ef-edfa8b6cef10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f523b529906f4fa989d0e836d6a08944)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:11.512963 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2767798-7d00-415a-a9ef-edfa8b6cef10 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:11.582708 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c2767798-7d00-415a-a9ef-edfa8b6cef10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f523b529906f4fa989d0e836d6a08944'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:11.597238 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c2767798-7d00-415a-a9ef-edfa8b6cef10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f523b529906f4fa989d0e836d6a08944'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:11.609931 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c2767798-7d00-415a-a9ef-edfa8b6cef10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f523b529906f4fa989d0e836d6a08944'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:11.618851 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c2767798-7d00-415a-a9ef-edfa8b6cef10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f523b529906f4fa989d0e836d6a08944'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:11.619723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c2767798-7d00-415a-a9ef-edfa8b6cef10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f523b529906f4fa989d0e836d6a08944'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:11.629178 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1393/2773] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:11 2022] DELETE /identity/v3/users/f523b529906f4fa989d0e836d6a08944 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:11.834823 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6e5d7-9dc3-4a7c-9aea-48ed95bbce93 service placement] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:11.845891 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b6e5d7-9dc3-4a7c-9aea-48ed95bbce93 service placement] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:11.920499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee3b16-78bd-4e70-832c-71b978f70b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:11.921388 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee3b16-78bd-4e70-832c-71b978f70b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:11.921680 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee3b16-78bd-4e70-832c-71b978f70b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:11.921964 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee3b16-78bd-4e70-832c-71b978f70b27 admin admin] PATH_INFO: `/v3/projects/1eb4b659032e4ec09a16c45b29d609f0` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:11.923374 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ee3b16-78bd-4e70-832c-71b978f70b27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eb4b659032e4ec09a16c45b29d609f0)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:11.925700 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ee3b16-78bd-4e70-832c-71b978f70b27 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:11.987480 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-35ee3b16-78bd-4e70-832c-71b978f70b27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eb4b659032e4ec09a16c45b29d609f0', 'request_id': 'req-35ee3b16-78bd-4e70-832c-71b978f70b27'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:11.997159 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-35ee3b16-78bd-4e70-832c-71b978f70b27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eb4b659032e4ec09a16c45b29d609f0', 'request_id': 'req-35ee3b16-78bd-4e70-832c-71b978f70b27'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:12.005518 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-35ee3b16-78bd-4e70-832c-71b978f70b27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eb4b659032e4ec09a16c45b29d609f0', 'request_id': 'req-35ee3b16-78bd-4e70-832c-71b978f70b27'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:12.020525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-35ee3b16-78bd-4e70-832c-71b978f70b27 admin admin] The token cache is being invalidate because project 1eb4b659032e4ec09a16c45b29d609f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:12.021246 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-35ee3b16-78bd-4e70-832c-71b978f70b27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:12.022149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-35ee3b16-78bd-4e70-832c-71b978f70b27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:12.024517 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1381/2774] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:11 2022] DELETE /identity/v3/projects/1eb4b659032e4ec09a16c45b29d609f0 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:12.028844 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2767798-7d00-415a-a9ef-edfa8b6cef10 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:12.031190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-c2767798-7d00-415a-a9ef-edfa8b6cef10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:12.045617 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2767798-7d00-415a-a9ef-edfa8b6cef10 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:12.095989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:12.119329 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7dfc7c-8f54-48e4-a464-7f0ad3d9582f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:12.120589 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dfc7c-8f54-48e4-a464-7f0ad3d9582f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:12.121048 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dfc7c-8f54-48e4-a464-7f0ad3d9582f admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:12.121523 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7dfc7c-8f54-48e4-a464-7f0ad3d9582f admin admin] PATH_INFO: `/v3/users/cc33294baa794bc3b67fb627e42bf3fc` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:12.122905 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7dfc7c-8f54-48e4-a464-7f0ad3d9582f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc33294baa794bc3b67fb627e42bf3fc)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:12.124891 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7dfc7c-8f54-48e4-a464-7f0ad3d9582f admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:12.180238 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-8d7dfc7c-8f54-48e4-a464-7f0ad3d9582f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc33294baa794bc3b67fb627e42bf3fc'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:12.196245 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-8d7dfc7c-8f54-48e4-a464-7f0ad3d9582f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc33294baa794bc3b67fb627e42bf3fc'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:12.207157 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-8d7dfc7c-8f54-48e4-a464-7f0ad3d9582f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc33294baa794bc3b67fb627e42bf3fc'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:12.215292 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-8d7dfc7c-8f54-48e4-a464-7f0ad3d9582f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc33294baa794bc3b67fb627e42bf3fc'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:12.216032 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-8d7dfc7c-8f54-48e4-a464-7f0ad3d9582f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc33294baa794bc3b67fb627e42bf3fc'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:12.223323 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1394/2775] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:12 2022] DELETE /identity/v3/users/cc33294baa794bc3b67fb627e42bf3fc => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:12.416119 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee3b16-78bd-4e70-832c-71b978f70b27 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:12.417620 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-35ee3b16-78bd-4e70-832c-71b978f70b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:12.437221 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee3b16-78bd-4e70-832c-71b978f70b27 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:12.510822 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74059034-1700-45cd-a65d-f871ec27e7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:12.511842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74059034-1700-45cd-a65d-f871ec27e7ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:12.512128 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74059034-1700-45cd-a65d-f871ec27e7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:12.512604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74059034-1700-45cd-a65d-f871ec27e7ec admin admin] PATH_INFO: `/v3/projects/d6d7cdd3e4cc418e9d1bb2280f92ae6d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:12.513967 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74059034-1700-45cd-a65d-f871ec27e7ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6d7cdd3e4cc418e9d1bb2280f92ae6d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:12.515958 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74059034-1700-45cd-a65d-f871ec27e7ec admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:12.517689 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:12.584768 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-74059034-1700-45cd-a65d-f871ec27e7ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6d7cdd3e4cc418e9d1bb2280f92ae6d', 'request_id': 'req-74059034-1700-45cd-a65d-f871ec27e7ec'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:12.594204 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-74059034-1700-45cd-a65d-f871ec27e7ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6d7cdd3e4cc418e9d1bb2280f92ae6d', 'request_id': 'req-74059034-1700-45cd-a65d-f871ec27e7ec'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:12.606291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-74059034-1700-45cd-a65d-f871ec27e7ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6d7cdd3e4cc418e9d1bb2280f92ae6d', 'request_id': 'req-74059034-1700-45cd-a65d-f871ec27e7ec'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:12.622879 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-74059034-1700-45cd-a65d-f871ec27e7ec admin admin] The token cache is being invalidate because project d6d7cdd3e4cc418e9d1bb2280f92ae6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:12.623233 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-74059034-1700-45cd-a65d-f871ec27e7ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:12.623909 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-74059034-1700-45cd-a65d-f871ec27e7ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:12.625493 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1382/2776] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:12 2022] DELETE /identity/v3/projects/d6d7cdd3e4cc418e9d1bb2280f92ae6d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:14.325629 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679550bd-3194-40a0-beaa-a7c60a738c6a None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:14.326279 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679550bd-3194-40a0-beaa-a7c60a738c6a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:14.326279 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679550bd-3194-40a0-beaa-a7c60a738c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:14.343405 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-679550bd-3194-40a0-beaa-a7c60a738c6a None None] MFA Rules not processed for user `6867c020bf2a4db3b0a4cebb32c64aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:14.344610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-679550bd-3194-40a0-beaa-a7c60a738c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:14.398955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1395/2777] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:47:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:47:14.649519 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74059034-1700-45cd-a65d-f871ec27e7ec admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:14.650969 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-74059034-1700-45cd-a65d-f871ec27e7ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:14.712505 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74059034-1700-45cd-a65d-f871ec27e7ec admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:14.716034 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03baa01e-8d29-4f7b-a8ea-9b2ac99c04b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:14.716669 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03baa01e-8d29-4f7b-a8ea-9b2ac99c04b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:14.716954 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03baa01e-8d29-4f7b-a8ea-9b2ac99c04b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:14.717230 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03baa01e-8d29-4f7b-a8ea-9b2ac99c04b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:14.718986 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-03baa01e-8d29-4f7b-a8ea-9b2ac99c04b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:14.729179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03baa01e-8d29-4f7b-a8ea-9b2ac99c04b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:14.730574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03baa01e-8d29-4f7b-a8ea-9b2ac99c04b7 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:14.739140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1383/2778] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:16.237789 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0ab6d-8cb2-49e2-8953-3ac91fc2d456 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:16.238272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0ab6d-8cb2-49e2-8953-3ac91fc2d456 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:16.238700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0ab6d-8cb2-49e2-8953-3ac91fc2d456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:16.257193 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-adf0ab6d-8cb2-49e2-8953-3ac91fc2d456 None None] MFA Rules not processed for user `df1aeb2be41c49819b811ecfa47d4155`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:16.258423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-adf0ab6d-8cb2-49e2-8953-3ac91fc2d456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:16.315648 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1396/2779] 10.222.0.43 () {58 vars in 1009 bytes} [Thu Jun 23 22:47:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:16.328769 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03baa01e-8d29-4f7b-a8ea-9b2ac99c04b7 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:16.330149 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-03baa01e-8d29-4f7b-a8ea-9b2ac99c04b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:16.402083 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03baa01e-8d29-4f7b-a8ea-9b2ac99c04b7 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:16.407140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e11415-9709-44aa-a24d-cd50e6d24f78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ce35c43e8294f4cbc880f2219a6eb76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:16.407799 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e11415-9709-44aa-a24d-cd50e6d24f78 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:16.408087 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e11415-9709-44aa-a24d-cd50e6d24f78 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:16.408384 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e11415-9709-44aa-a24d-cd50e6d24f78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:16.416778 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e11415-9709-44aa-a24d-cd50e6d24f78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:16.418617 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e11415-9709-44aa-a24d-cd50e6d24f78 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:16.427937 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1384/2780] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 22:47:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:47:26.308657 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=84,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:26.423773 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:44.164689 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0ab6d-8cb2-49e2-8953-3ac91fc2d456 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:44.174634 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0ab6d-8cb2-49e2-8953-3ac91fc2d456 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:44.178229 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e7e44-c58d-4495-9605-25abd24ff279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:44.178809 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e7e44-c58d-4495-9605-25abd24ff279 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:44.179129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e7e44-c58d-4495-9605-25abd24ff279 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:44.179412 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e7e44-c58d-4495-9605-25abd24ff279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:44.181190 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8a1e7e44-c58d-4495-9605-25abd24ff279 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:44.241418 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1e7e44-c58d-4495-9605-25abd24ff279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:44.242861 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1e7e44-c58d-4495-9605-25abd24ff279 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:44.252672 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1397/2781] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:46.217179 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e11415-9709-44aa-a24d-cd50e6d24f78 service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:46.219536 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b9e11415-9709-44aa-a24d-cd50e6d24f78 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:46.244011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e11415-9709-44aa-a24d-cd50e6d24f78 service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:46.252461 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59332d6-3716-484d-b72c-7c3b589022d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:46.253315 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59332d6-3716-484d-b72c-7c3b589022d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:46.254464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59332d6-3716-484d-b72c-7c3b589022d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:46.254731 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59332d6-3716-484d-b72c-7c3b589022d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:46.258585 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-b59332d6-3716-484d-b72c-7c3b589022d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:46.362332 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59332d6-3716-484d-b72c-7c3b589022d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:46.364533 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59332d6-3716-484d-b72c-7c3b589022d5 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:46.380729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1385/2782] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:49.939975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e7e44-c58d-4495-9605-25abd24ff279 service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:49.941449 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8a1e7e44-c58d-4495-9605-25abd24ff279 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:49.960565 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e7e44-c58d-4495-9605-25abd24ff279 service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:49.965465 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99cafb5-fcbe-4cef-848b-39f0c54e8e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:49.966359 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99cafb5-fcbe-4cef-848b-39f0c54e8e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:49.966645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99cafb5-fcbe-4cef-848b-39f0c54e8e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:49.966972 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99cafb5-fcbe-4cef-848b-39f0c54e8e42 admin admin] PATH_INFO: `/v3/users/1a93ea9f0a3f47d68e53c464c1d52c92` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:49.968424 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99cafb5-fcbe-4cef-848b-39f0c54e8e42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a93ea9f0a3f47d68e53c464c1d52c92)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:49.970450 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99cafb5-fcbe-4cef-848b-39f0c54e8e42 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:50.025837 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-a99cafb5-fcbe-4cef-848b-39f0c54e8e42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a93ea9f0a3f47d68e53c464c1d52c92'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:50.037675 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-a99cafb5-fcbe-4cef-848b-39f0c54e8e42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a93ea9f0a3f47d68e53c464c1d52c92'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:50.045287 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-a99cafb5-fcbe-4cef-848b-39f0c54e8e42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a93ea9f0a3f47d68e53c464c1d52c92'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:50.052892 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-a99cafb5-fcbe-4cef-848b-39f0c54e8e42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a93ea9f0a3f47d68e53c464c1d52c92'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:50.053528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-a99cafb5-fcbe-4cef-848b-39f0c54e8e42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a93ea9f0a3f47d68e53c464c1d52c92'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:50.059608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1398/2783] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:49 2022] DELETE /identity/v3/users/1a93ea9f0a3f47d68e53c464c1d52c92 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:50.231018 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59332d6-3716-484d-b72c-7c3b589022d5 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:50.241206 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59332d6-3716-484d-b72c-7c3b589022d5 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:50.301857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc0319-5993-40f6-8ed8-be1fb3d4142b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:50.302721 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc0319-5993-40f6-8ed8-be1fb3d4142b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:50.303036 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc0319-5993-40f6-8ed8-be1fb3d4142b admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:50.303323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc0319-5993-40f6-8ed8-be1fb3d4142b admin admin] PATH_INFO: `/v3/projects/9ab3a2e6c9634a06b33f1d82a28dc3f1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:50.304506 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc0319-5993-40f6-8ed8-be1fb3d4142b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ab3a2e6c9634a06b33f1d82a28dc3f1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:50.306380 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc0319-5993-40f6-8ed8-be1fb3d4142b admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:50.353572 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bdcc0319-5993-40f6-8ed8-be1fb3d4142b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ab3a2e6c9634a06b33f1d82a28dc3f1', 'request_id': 'req-bdcc0319-5993-40f6-8ed8-be1fb3d4142b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:50.361772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bdcc0319-5993-40f6-8ed8-be1fb3d4142b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ab3a2e6c9634a06b33f1d82a28dc3f1', 'request_id': 'req-bdcc0319-5993-40f6-8ed8-be1fb3d4142b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:50.367675 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bdcc0319-5993-40f6-8ed8-be1fb3d4142b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ab3a2e6c9634a06b33f1d82a28dc3f1', 'request_id': 'req-bdcc0319-5993-40f6-8ed8-be1fb3d4142b'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:50.379077 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bdcc0319-5993-40f6-8ed8-be1fb3d4142b admin admin] The token cache is being invalidate because project 9ab3a2e6c9634a06b33f1d82a28dc3f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:50.379377 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bdcc0319-5993-40f6-8ed8-be1fb3d4142b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:50.380020 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-bdcc0319-5993-40f6-8ed8-be1fb3d4142b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:50.381555 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1386/2784] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:50 2022] DELETE /identity/v3/projects/9ab3a2e6c9634a06b33f1d82a28dc3f1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:50.386480 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5d35c-971e-4f09-b4ef-3b154dde569c None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:50.386782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5d35c-971e-4f09-b4ef-3b154dde569c None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:50.387116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5d35c-971e-4f09-b4ef-3b154dde569c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:50.405128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-1dc5d35c-971e-4f09-b4ef-3b154dde569c None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:50.406331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-1dc5d35c-971e-4f09-b4ef-3b154dde569c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:50.467412 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1399/2785] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:50.477092 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7d3f9-c29e-463b-a164-d7c2e49c590b None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:50.477400 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7d3f9-c29e-463b-a164-d7c2e49c590b None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:50.477661 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7d3f9-c29e-463b-a164-d7c2e49c590b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:50.497466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8de7d3f9-c29e-463b-a164-d7c2e49c590b None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:50.498684 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8de7d3f9-c29e-463b-a164-d7c2e49c590b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:50.506690 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1387/2786] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:47:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:50.510869 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5d35c-971e-4f09-b4ef-3b154dde569c None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:50.521067 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5d35c-971e-4f09-b4ef-3b154dde569c None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:50.525659 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d781f4d6-8a39-4567-a9aa-dff628771d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:50.526317 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781f4d6-8a39-4567-a9aa-dff628771d82 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:50.526608 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781f4d6-8a39-4567-a9aa-dff628771d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:50.526894 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781f4d6-8a39-4567-a9aa-dff628771d82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:50.527654 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d781f4d6-8a39-4567-a9aa-dff628771d82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:47:50.527967 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d781f4d6-8a39-4567-a9aa-dff628771d82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:50.529572 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d781f4d6-8a39-4567-a9aa-dff628771d82 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:50.531087 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:50.544137 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1400/2787] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:47:50.548442 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7d3f9-c29e-463b-a164-d7c2e49c590b None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:50.557556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7d3f9-c29e-463b-a164-d7c2e49c590b None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:50.562135 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e5d7f-d78e-4a11-a191-a51dbe23c008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:50.562776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e5d7f-d78e-4a11-a191-a51dbe23c008 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:50.563113 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e5d7f-d78e-4a11-a191-a51dbe23c008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:50.563424 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e5d7f-d78e-4a11-a191-a51dbe23c008 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:50.564181 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e5d7f-d78e-4a11-a191-a51dbe23c008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:50.566049 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e5d7f-d78e-4a11-a191-a51dbe23c008 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:50.579981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:50.593302 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1388/2788] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:50 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:47:50.598047 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d781f4d6-8a39-4567-a9aa-dff628771d82 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:50.608469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d781f4d6-8a39-4567-a9aa-dff628771d82 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:50.667120 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be19bf-1f64-454f-8e54-bad3df7a2b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:50.667813 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be19bf-1f64-454f-8e54-bad3df7a2b7f admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:50.668102 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be19bf-1f64-454f-8e54-bad3df7a2b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:50.668644 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be19bf-1f64-454f-8e54-bad3df7a2b7f admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:50.669337 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be19bf-1f64-454f-8e54-bad3df7a2b7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:50.671317 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be19bf-1f64-454f-8e54-bad3df7a2b7f admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:50.692330 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1401/2789] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:50 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:50.696484 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e5d7f-d78e-4a11-a191-a51dbe23c008 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:50.708629 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e5d7f-d78e-4a11-a191-a51dbe23c008 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:50.713254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448ac99-9a0e-4ae9-bf53-43f68c62c087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:50.714107 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448ac99-9a0e-4ae9-bf53-43f68c62c087 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:50.714394 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448ac99-9a0e-4ae9-bf53-43f68c62c087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:50.714679 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448ac99-9a0e-4ae9-bf53-43f68c62c087 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:50.715444 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448ac99-9a0e-4ae9-bf53-43f68c62c087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:50.717027 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448ac99-9a0e-4ae9-bf53-43f68c62c087 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:50.731080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1389/2790] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:47:50.733566 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be19bf-1f64-454f-8e54-bad3df7a2b7f admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:50.744504 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be19bf-1f64-454f-8e54-bad3df7a2b7f admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:50.749165 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fa6e0-3bb4-42ef-91db-566fbc30d5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:50.749830 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fa6e0-3bb4-42ef-91db-566fbc30d5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:50.750119 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fa6e0-3bb4-42ef-91db-566fbc30d5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:50.750401 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fa6e0-3bb4-42ef-91db-566fbc30d5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:50.751768 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fa6e0-3bb4-42ef-91db-566fbc30d5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:50.753367 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fa6e0-3bb4-42ef-91db-566fbc30d5e6 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:50.763113 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1402/2791] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:50.767877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448ac99-9a0e-4ae9-bf53-43f68c62c087 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:50.778017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448ac99-9a0e-4ae9-bf53-43f68c62c087 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:50.783310 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8e62b-809d-4150-8c89-2afdad420e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:50.784181 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8e62b-809d-4150-8c89-2afdad420e2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:50.784464 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8e62b-809d-4150-8c89-2afdad420e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:50.784741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8e62b-809d-4150-8c89-2afdad420e2c admin admin] PATH_INFO: `/v3/projects/3d1d101eeb854a64afde11b2c6391db0/users/b6cfd190f59c444ebdb5e23794e6b21e/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:50.799211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8e62b-809d-4150-8c89-2afdad420e2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d1d101eeb854a64afde11b2c6391db0, user_id=b6cfd190f59c444ebdb5e23794e6b21e, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:50.801806 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8e62b-809d-4150-8c89-2afdad420e2c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:50.814480 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1390/2792] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:50 2022] PUT /identity/v3/projects/3d1d101eeb854a64afde11b2c6391db0/users/b6cfd190f59c444ebdb5e23794e6b21e/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:50.825231 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d769f-f453-49b3-93c6-955346f4231a None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:50.825527 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d769f-f453-49b3-93c6-955346f4231a None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:50.825813 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d769f-f453-49b3-93c6-955346f4231a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:50.855724 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-5d4d769f-f453-49b3-93c6-955346f4231a None None] MFA Rules not processed for user `b6cfd190f59c444ebdb5e23794e6b21e`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:50.859204 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-5d4d769f-f453-49b3-93c6-955346f4231a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:50.927870 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1403/2793] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:47:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:50.932431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8e62b-809d-4150-8c89-2afdad420e2c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:50.992940 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8e62b-809d-4150-8c89-2afdad420e2c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:50.997574 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c46fd-6da2-4b06-b34a-e6116db76696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:50.998228 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c46fd-6da2-4b06-b34a-e6116db76696 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:50.998510 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c46fd-6da2-4b06-b34a-e6116db76696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:50.998786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c46fd-6da2-4b06-b34a-e6116db76696 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:50.999501 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c46fd-6da2-4b06-b34a-e6116db76696 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:51.002201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c46fd-6da2-4b06-b34a-e6116db76696 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:51.026707 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1391/2794] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:50 2022] POST /identity/v3/projects => generated 363 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:51.030860 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d769f-f453-49b3-93c6-955346f4231a None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:51.041531 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4d769f-f453-49b3-93c6-955346f4231a None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:51.096812 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b367a1f-d003-4351-bafa-d509f3608e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:51.097478 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b367a1f-d003-4351-bafa-d509f3608e2c admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:51.097764 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b367a1f-d003-4351-bafa-d509f3608e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:51.098040 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b367a1f-d003-4351-bafa-d509f3608e2c admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:51.098715 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b367a1f-d003-4351-bafa-d509f3608e2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:51.100645 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b367a1f-d003-4351-bafa-d509f3608e2c admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:51.107991 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:51.123563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1404/2795] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:51 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:51.128399 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c46fd-6da2-4b06-b34a-e6116db76696 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:51.141126 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c46fd-6da2-4b06-b34a-e6116db76696 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:51.146462 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02553802-d237-4e59-829a-8a305f1018b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:51.147167 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02553802-d237-4e59-829a-8a305f1018b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:51.147458 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02553802-d237-4e59-829a-8a305f1018b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:51.147767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02553802-d237-4e59-829a-8a305f1018b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:51.148495 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02553802-d237-4e59-829a-8a305f1018b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:51.150094 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02553802-d237-4e59-829a-8a305f1018b8 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:51.160988 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1392/2796] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:51.165170 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b367a1f-d003-4351-bafa-d509f3608e2c admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:51.175663 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b367a1f-d003-4351-bafa-d509f3608e2c admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:51.180252 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9990305-9635-485c-97b2-b1ad961a78de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:51.181129 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9990305-9635-485c-97b2-b1ad961a78de admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:51.181414 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9990305-9635-485c-97b2-b1ad961a78de admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:51.181696 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9990305-9635-485c-97b2-b1ad961a78de admin admin] PATH_INFO: `/v3/projects/9dce2daaf0e74f7da73916506670c48b/users/36d40c6c3f3c4e52a6f392eea2d54a03/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:51.197273 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9990305-9635-485c-97b2-b1ad961a78de admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dce2daaf0e74f7da73916506670c48b, user_id=36d40c6c3f3c4e52a6f392eea2d54a03, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:51.199989 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9990305-9635-485c-97b2-b1ad961a78de admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:51.213731 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1405/2797] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:51 2022] PUT /identity/v3/projects/9dce2daaf0e74f7da73916506670c48b/users/36d40c6c3f3c4e52a6f392eea2d54a03/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:51.217757 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02553802-d237-4e59-829a-8a305f1018b8 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:51.228825 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02553802-d237-4e59-829a-8a305f1018b8 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:51.285479 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27ce67-123a-416b-9513-25517231e4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:51.286159 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27ce67-123a-416b-9513-25517231e4ce admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:51.286349 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27ce67-123a-416b-9513-25517231e4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:51.286633 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27ce67-123a-416b-9513-25517231e4ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:51.287391 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27ce67-123a-416b-9513-25517231e4ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:51.288974 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27ce67-123a-416b-9513-25517231e4ce admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:51.299479 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1393/2798] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:51 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:51.303775 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9990305-9635-485c-97b2-b1ad961a78de admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:51.314594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9990305-9635-485c-97b2-b1ad961a78de admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:51.319215 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cc1ea-5039-41f9-ad22-ac28ba7a0cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:51.320098 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cc1ea-5039-41f9-ad22-ac28ba7a0cf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:51.320386 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cc1ea-5039-41f9-ad22-ac28ba7a0cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:51.320670 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cc1ea-5039-41f9-ad22-ac28ba7a0cf5 admin admin] PATH_INFO: `/v3/projects/9dce2daaf0e74f7da73916506670c48b/users/36d40c6c3f3c4e52a6f392eea2d54a03/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:51.322621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23cc1ea-5039-41f9-ad22-ac28ba7a0cf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dce2daaf0e74f7da73916506670c48b, user_id=36d40c6c3f3c4e52a6f392eea2d54a03, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:51.325280 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23cc1ea-5039-41f9-ad22-ac28ba7a0cf5 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:51.335959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1406/2799] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:51 2022] PUT /identity/v3/projects/9dce2daaf0e74f7da73916506670c48b/users/36d40c6c3f3c4e52a6f392eea2d54a03/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:51.345941 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c8dfd-315e-42ee-9972-e4483881cda9 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:51.346237 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c8dfd-315e-42ee-9972-e4483881cda9 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:51.346526 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0c8dfd-315e-42ee-9972-e4483881cda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:51.372366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-2a0c8dfd-315e-42ee-9972-e4483881cda9 None None] MFA Rules not processed for user `36d40c6c3f3c4e52a6f392eea2d54a03`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:51.373566 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-2a0c8dfd-315e-42ee-9972-e4483881cda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:51.392171 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:51.441743 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1394/2800] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:47:51 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:51.452823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cc1ea-5039-41f9-ad22-ac28ba7a0cf5 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:51.454853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-e23cc1ea-5039-41f9-ad22-ac28ba7a0cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:51.511785 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cc1ea-5039-41f9-ad22-ac28ba7a0cf5 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:51.515448 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25978e7-b770-4389-b192-6cc90cf5142d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:51.516106 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25978e7-b770-4389-b192-6cc90cf5142d service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:51.516396 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25978e7-b770-4389-b192-6cc90cf5142d service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:51.516683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25978e7-b770-4389-b192-6cc90cf5142d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:51.525798 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25978e7-b770-4389-b192-6cc90cf5142d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:51.527286 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25978e7-b770-4389-b192-6cc90cf5142d service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:51.537098 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1407/2801] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:51 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:51.608815 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c8dfd-315e-42ee-9972-e4483881cda9 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:51.618986 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0c8dfd-315e-42ee-9972-e4483881cda9 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:51.622551 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd19ae-2601-4be9-80ed-bcb78ba6b64b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:51.623220 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd19ae-2601-4be9-80ed-bcb78ba6b64b service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:51.623507 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd19ae-2601-4be9-80ed-bcb78ba6b64b service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:51.623785 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd19ae-2601-4be9-80ed-bcb78ba6b64b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:51.681265 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd19ae-2601-4be9-80ed-bcb78ba6b64b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:51.682988 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd19ae-2601-4be9-80ed-bcb78ba6b64b service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:51.692389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1395/2802] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:51 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:54.378062 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25978e7-b770-4389-b192-6cc90cf5142d service cinder] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:54.441876 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25978e7-b770-4389-b192-6cc90cf5142d service cinder] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:54.446428 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8551cf-d342-479b-a3bb-fbba2496834e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:54.447366 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8551cf-d342-479b-a3bb-fbba2496834e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:54.447658 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8551cf-d342-479b-a3bb-fbba2496834e admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:54.447956 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8551cf-d342-479b-a3bb-fbba2496834e admin admin] PATH_INFO: `/v3/users/b6cfd190f59c444ebdb5e23794e6b21e` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:54.449130 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8551cf-d342-479b-a3bb-fbba2496834e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6cfd190f59c444ebdb5e23794e6b21e)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:54.451080 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8551cf-d342-479b-a3bb-fbba2496834e admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:54.497616 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-bb8551cf-d342-479b-a3bb-fbba2496834e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6cfd190f59c444ebdb5e23794e6b21e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:54.506028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:54.510957 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-bb8551cf-d342-479b-a3bb-fbba2496834e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6cfd190f59c444ebdb5e23794e6b21e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:54.519128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-bb8551cf-d342-479b-a3bb-fbba2496834e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6cfd190f59c444ebdb5e23794e6b21e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:54.529037 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-bb8551cf-d342-479b-a3bb-fbba2496834e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6cfd190f59c444ebdb5e23794e6b21e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:54.531470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-bb8551cf-d342-479b-a3bb-fbba2496834e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6cfd190f59c444ebdb5e23794e6b21e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:54.540096 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1408/2803] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:54 2022] DELETE /identity/v3/users/b6cfd190f59c444ebdb5e23794e6b21e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:54.545366 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30cfa5-a71e-4d8c-8617-0f03373f8dc2 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:54.545872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30cfa5-a71e-4d8c-8617-0f03373f8dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:54.546534 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30cfa5-a71e-4d8c-8617-0f03373f8dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:54.565647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-8b30cfa5-a71e-4d8c-8617-0f03373f8dc2 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:54.566900 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8b30cfa5-a71e-4d8c-8617-0f03373f8dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:54.625233 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1396/2804] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:54.634895 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8551cf-d342-479b-a3bb-fbba2496834e admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:54.636395 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-bb8551cf-d342-479b-a3bb-fbba2496834e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:54.692868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8551cf-d342-479b-a3bb-fbba2496834e admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:54.696516 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ea246-1fd5-42eb-8595-dff556567584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:54.697166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ea246-1fd5-42eb-8595-dff556567584 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:54.697455 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ea246-1fd5-42eb-8595-dff556567584 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:54.697739 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ea246-1fd5-42eb-8595-dff556567584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:54.707192 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ea246-1fd5-42eb-8595-dff556567584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:54.708615 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ea246-1fd5-42eb-8595-dff556567584 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:54.717922 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1409/2805] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:54.990612 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30cfa5-a71e-4d8c-8617-0f03373f8dc2 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:54.992080 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-8b30cfa5-a71e-4d8c-8617-0f03373f8dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:55.004970 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30cfa5-a71e-4d8c-8617-0f03373f8dc2 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:55.009760 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2ca99-fa9a-4adb-8b92-43d8acf299ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:55.010686 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2ca99-fa9a-4adb-8b92-43d8acf299ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:55.011014 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2ca99-fa9a-4adb-8b92-43d8acf299ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:55.011303 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2ca99-fa9a-4adb-8b92-43d8acf299ef admin admin] PATH_INFO: `/v3/projects/3d1d101eeb854a64afde11b2c6391db0` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:55.012494 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a2ca99-fa9a-4adb-8b92-43d8acf299ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d1d101eeb854a64afde11b2c6391db0)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:55.014501 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a2ca99-fa9a-4adb-8b92-43d8acf299ef admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:55.061559 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-32a2ca99-fa9a-4adb-8b92-43d8acf299ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d1d101eeb854a64afde11b2c6391db0', 'request_id': 'req-32a2ca99-fa9a-4adb-8b92-43d8acf299ef'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:55.063141 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:55.070192 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-32a2ca99-fa9a-4adb-8b92-43d8acf299ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d1d101eeb854a64afde11b2c6391db0', 'request_id': 'req-32a2ca99-fa9a-4adb-8b92-43d8acf299ef'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:55.076310 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-32a2ca99-fa9a-4adb-8b92-43d8acf299ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d1d101eeb854a64afde11b2c6391db0', 'request_id': 'req-32a2ca99-fa9a-4adb-8b92-43d8acf299ef'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:55.088024 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-32a2ca99-fa9a-4adb-8b92-43d8acf299ef admin admin] The token cache is being invalidate because project 3d1d101eeb854a64afde11b2c6391db0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:55.088313 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-32a2ca99-fa9a-4adb-8b92-43d8acf299ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:55.088966 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-32a2ca99-fa9a-4adb-8b92-43d8acf299ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:55.090494 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1397/2806] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:54 2022] DELETE /identity/v3/projects/3d1d101eeb854a64afde11b2c6391db0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:55.094583 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ea246-1fd5-42eb-8595-dff556567584 service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:55.105079 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ea246-1fd5-42eb-8595-dff556567584 service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:55.167043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483171af-dc9c-4862-8f8d-dfbe5be57ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:55.167925 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483171af-dc9c-4862-8f8d-dfbe5be57ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:55.168469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483171af-dc9c-4862-8f8d-dfbe5be57ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:55.168756 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483171af-dc9c-4862-8f8d-dfbe5be57ca0 admin admin] PATH_INFO: `/v3/users/36d40c6c3f3c4e52a6f392eea2d54a03` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:55.169939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483171af-dc9c-4862-8f8d-dfbe5be57ca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36d40c6c3f3c4e52a6f392eea2d54a03)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:55.171897 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483171af-dc9c-4862-8f8d-dfbe5be57ca0 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:55.217530 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-483171af-dc9c-4862-8f8d-dfbe5be57ca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36d40c6c3f3c4e52a6f392eea2d54a03'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:55.230176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-483171af-dc9c-4862-8f8d-dfbe5be57ca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36d40c6c3f3c4e52a6f392eea2d54a03'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:55.238237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-483171af-dc9c-4862-8f8d-dfbe5be57ca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36d40c6c3f3c4e52a6f392eea2d54a03'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:55.245164 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-483171af-dc9c-4862-8f8d-dfbe5be57ca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36d40c6c3f3c4e52a6f392eea2d54a03'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:55.245800 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-483171af-dc9c-4862-8f8d-dfbe5be57ca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36d40c6c3f3c4e52a6f392eea2d54a03'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:55.247188 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:55.253306 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1410/2807] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:55 2022] DELETE /identity/v3/users/36d40c6c3f3c4e52a6f392eea2d54a03 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:55.520526 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2ca99-fa9a-4adb-8b92-43d8acf299ef admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:55.522011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-32a2ca99-fa9a-4adb-8b92-43d8acf299ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:55.536060 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2ca99-fa9a-4adb-8b92-43d8acf299ef admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:55.592389 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ba43c-619c-4e00-bc29-50a3b29d21a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:55.593253 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ba43c-619c-4e00-bc29-50a3b29d21a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:55.593542 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ba43c-619c-4e00-bc29-50a3b29d21a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:55.593827 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ba43c-619c-4e00-bc29-50a3b29d21a7 admin admin] PATH_INFO: `/v3/projects/9dce2daaf0e74f7da73916506670c48b` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:55.595646 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ba43c-619c-4e00-bc29-50a3b29d21a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dce2daaf0e74f7da73916506670c48b)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:55.597478 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717ba43c-619c-4e00-bc29-50a3b29d21a7 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:55.644010 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-717ba43c-619c-4e00-bc29-50a3b29d21a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dce2daaf0e74f7da73916506670c48b', 'request_id': 'req-717ba43c-619c-4e00-bc29-50a3b29d21a7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:55.651433 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-717ba43c-619c-4e00-bc29-50a3b29d21a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dce2daaf0e74f7da73916506670c48b', 'request_id': 'req-717ba43c-619c-4e00-bc29-50a3b29d21a7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:55.656851 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-717ba43c-619c-4e00-bc29-50a3b29d21a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dce2daaf0e74f7da73916506670c48b', 'request_id': 'req-717ba43c-619c-4e00-bc29-50a3b29d21a7'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:55.668595 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-717ba43c-619c-4e00-bc29-50a3b29d21a7 admin admin] The token cache is being invalidate because project 9dce2daaf0e74f7da73916506670c48b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:55.668887 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-717ba43c-619c-4e00-bc29-50a3b29d21a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:55.669525 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-717ba43c-619c-4e00-bc29-50a3b29d21a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:55.671079 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1398/2808] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:55 2022] DELETE /identity/v3/projects/9dce2daaf0e74f7da73916506670c48b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:55.675961 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4e2a0-f94f-48e6-b3ef-2286035e5d46 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:55.676525 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4e2a0-f94f-48e6-b3ef-2286035e5d46 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:55.676831 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4e2a0-f94f-48e6-b3ef-2286035e5d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:55.694203 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-94a4e2a0-f94f-48e6-b3ef-2286035e5d46 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:55.695473 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-94a4e2a0-f94f-48e6-b3ef-2286035e5d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:55.755272 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1411/2809] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:47:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:55.763062 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5527a-6c16-4c75-9d51-65e58a6434a1 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:55.763360 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5527a-6c16-4c75-9d51-65e58a6434a1 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:55.763644 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5527a-6c16-4c75-9d51-65e58a6434a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:55.782508 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-cac5527a-6c16-4c75-9d51-65e58a6434a1 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:55.784204 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cac5527a-6c16-4c75-9d51-65e58a6434a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:55.792017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1399/2810] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:47:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:55.796390 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4e2a0-f94f-48e6-b3ef-2286035e5d46 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:55.807023 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4e2a0-f94f-48e6-b3ef-2286035e5d46 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:55.811553 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fa77e-6f80-4f50-bc11-143707b9bd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:55.812234 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fa77e-6f80-4f50-bc11-143707b9bd75 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:55.812522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fa77e-6f80-4f50-bc11-143707b9bd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:55.812802 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fa77e-6f80-4f50-bc11-143707b9bd75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:55.813653 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3fa77e-6f80-4f50-bc11-143707b9bd75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80201) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:47:55.814082 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3fa77e-6f80-4f50-bc11-143707b9bd75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:55.815757 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3fa77e-6f80-4f50-bc11-143707b9bd75 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:55.826980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1412/2811] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:47:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:47:55.832842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5527a-6c16-4c75-9d51-65e58a6434a1 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:55.842042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5527a-6c16-4c75-9d51-65e58a6434a1 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:55.847733 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c837f-cf74-48ac-a1f5-ef540839e1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:55.848322 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c837f-cf74-48ac-a1f5-ef540839e1d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:55.848608 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c837f-cf74-48ac-a1f5-ef540839e1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:55.848888 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c837f-cf74-48ac-a1f5-ef540839e1d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:55.849580 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c837f-cf74-48ac-a1f5-ef540839e1d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:55.851488 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c837f-cf74-48ac-a1f5-ef540839e1d6 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:55.877886 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1400/2812] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:55 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:55.882036 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fa77e-6f80-4f50-bc11-143707b9bd75 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:55.892399 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fa77e-6f80-4f50-bc11-143707b9bd75 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:55.949255 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c425416-01ea-4795-bb0b-e3ab7f4ae04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:55.949927 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c425416-01ea-4795-bb0b-e3ab7f4ae04c admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:55.950216 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c425416-01ea-4795-bb0b-e3ab7f4ae04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:55.950501 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c425416-01ea-4795-bb0b-e3ab7f4ae04c admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:55.951234 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c425416-01ea-4795-bb0b-e3ab7f4ae04c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:55.953149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c425416-01ea-4795-bb0b-e3ab7f4ae04c admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:55.974622 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1413/2813] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:55 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:55.978908 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c837f-cf74-48ac-a1f5-ef540839e1d6 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:55.991767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c837f-cf74-48ac-a1f5-ef540839e1d6 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:55.996565 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce7740-241b-4240-98ed-7a698d5e879a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:55.997871 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce7740-241b-4240-98ed-7a698d5e879a admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:55.998663 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce7740-241b-4240-98ed-7a698d5e879a admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:55.998663 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce7740-241b-4240-98ed-7a698d5e879a admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:55.999478 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce7740-241b-4240-98ed-7a698d5e879a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:56.001748 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce7740-241b-4240-98ed-7a698d5e879a admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:56.013399 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1401/2814] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:56.017924 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c425416-01ea-4795-bb0b-e3ab7f4ae04c admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:56.029163 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c425416-01ea-4795-bb0b-e3ab7f4ae04c admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:56.033984 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43b305-6af1-4713-ad29-06accef2ee7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:56.034692 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43b305-6af1-4713-ad29-06accef2ee7a admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:56.035013 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43b305-6af1-4713-ad29-06accef2ee7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:56.035304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43b305-6af1-4713-ad29-06accef2ee7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:56.036043 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43b305-6af1-4713-ad29-06accef2ee7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:56.037660 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43b305-6af1-4713-ad29-06accef2ee7a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:56.047983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1414/2815] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 22:47:56.052186 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce7740-241b-4240-98ed-7a698d5e879a admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:56.062238 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce7740-241b-4240-98ed-7a698d5e879a admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:56.067489 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460c890-3185-4e39-b51d-3bdeb9723ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:56.068370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460c890-3185-4e39-b51d-3bdeb9723ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:56.068656 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460c890-3185-4e39-b51d-3bdeb9723ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:56.068938 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460c890-3185-4e39-b51d-3bdeb9723ac3 admin admin] PATH_INFO: `/v3/projects/04d77659ad9d429bab8449440909d4e5/users/3d0b100b4d3d4758a0375978e2a01a7e/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:56.084890 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7460c890-3185-4e39-b51d-3bdeb9723ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04d77659ad9d429bab8449440909d4e5, user_id=3d0b100b4d3d4758a0375978e2a01a7e, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:56.087568 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7460c890-3185-4e39-b51d-3bdeb9723ac3 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:56.100627 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1402/2816] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:56 2022] PUT /identity/v3/projects/04d77659ad9d429bab8449440909d4e5/users/3d0b100b4d3d4758a0375978e2a01a7e/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:56.110818 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9525f-5135-4d52-a591-44b456769c32 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:56.111151 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9525f-5135-4d52-a591-44b456769c32 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:56.111440 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9525f-5135-4d52-a591-44b456769c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:56.138662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-3af9525f-5135-4d52-a591-44b456769c32 None None] MFA Rules not processed for user `3d0b100b4d3d4758a0375978e2a01a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:56.139931 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3af9525f-5135-4d52-a591-44b456769c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:56.195345 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:56.206717 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1415/2817] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:47:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:56.211229 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460c890-3185-4e39-b51d-3bdeb9723ac3 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:56.216839 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:56.275948 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460c890-3185-4e39-b51d-3bdeb9723ac3 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:56.280658 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6565b4-0090-484a-bc87-d05670ce15b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:56.281316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6565b4-0090-484a-bc87-d05670ce15b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:56.281609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6565b4-0090-484a-bc87-d05670ce15b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:56.281893 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6565b4-0090-484a-bc87-d05670ce15b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:56.282609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6565b4-0090-484a-bc87-d05670ce15b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:56.284529 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6565b4-0090-484a-bc87-d05670ce15b1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:56.309622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1403/2818] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:56 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:56.314078 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9525f-5135-4d52-a591-44b456769c32 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:56.324732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9525f-5135-4d52-a591-44b456769c32 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:56.381161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd565a9d-cc72-4779-b03a-7abe84d53d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:56.381839 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd565a9d-cc72-4779-b03a-7abe84d53d74 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:56.382132 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd565a9d-cc72-4779-b03a-7abe84d53d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:56.382415 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd565a9d-cc72-4779-b03a-7abe84d53d74 admin admin] PATH_INFO: `/v3/users` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:56.383140 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd565a9d-cc72-4779-b03a-7abe84d53d74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:56.385048 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd565a9d-cc72-4779-b03a-7abe84d53d74 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:56.404804 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1416/2819] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:56 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:56.408991 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6565b4-0090-484a-bc87-d05670ce15b1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:56.419609 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6565b4-0090-484a-bc87-d05670ce15b1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:56.424772 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e174c-bd48-4d14-855e-457e1a601fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:56.425434 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e174c-bd48-4d14-855e-457e1a601fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:56.425722 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e174c-bd48-4d14-855e-457e1a601fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:56.426017 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e174c-bd48-4d14-855e-457e1a601fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:56.426738 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e174c-bd48-4d14-855e-457e1a601fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:56.428372 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e174c-bd48-4d14-855e-457e1a601fe9 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:56.438791 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1404/2820] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:56.442946 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd565a9d-cc72-4779-b03a-7abe84d53d74 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:56.452746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd565a9d-cc72-4779-b03a-7abe84d53d74 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:56.457794 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd972317-0eda-4e83-a1ad-674d9129399a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:56.458665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd972317-0eda-4e83-a1ad-674d9129399a admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:56.458983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd972317-0eda-4e83-a1ad-674d9129399a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:56.459383 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd972317-0eda-4e83-a1ad-674d9129399a admin admin] PATH_INFO: `/v3/projects/8850424d4c4945dfb02012c49ca545df/users/fc355431025e4cdaa165de0d67e4ef61/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:56.474226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd972317-0eda-4e83-a1ad-674d9129399a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8850424d4c4945dfb02012c49ca545df, user_id=fc355431025e4cdaa165de0d67e4ef61, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:56.476904 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd972317-0eda-4e83-a1ad-674d9129399a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:56.489627 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1417/2821] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:56 2022] PUT /identity/v3/projects/8850424d4c4945dfb02012c49ca545df/users/fc355431025e4cdaa165de0d67e4ef61/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:56.493596 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e174c-bd48-4d14-855e-457e1a601fe9 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:56.504197 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e174c-bd48-4d14-855e-457e1a601fe9 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:56.561630 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfe081-2409-4b0f-9653-ff719b842fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:56.562293 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfe081-2409-4b0f-9653-ff719b842fda admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:56.562585 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfe081-2409-4b0f-9653-ff719b842fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:56.562870 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfe081-2409-4b0f-9653-ff719b842fda admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:56.564285 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfe081-2409-4b0f-9653-ff719b842fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:56.565881 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfe081-2409-4b0f-9653-ff719b842fda admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:56.575622 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1405/2822] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:56.580588 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd972317-0eda-4e83-a1ad-674d9129399a admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:56.590615 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd972317-0eda-4e83-a1ad-674d9129399a admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:56.595586 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24395765-2aaa-4eec-b7b5-8ab06c8ebe21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:56.596531 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24395765-2aaa-4eec-b7b5-8ab06c8ebe21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:56.596820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24395765-2aaa-4eec-b7b5-8ab06c8ebe21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:56.597105 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24395765-2aaa-4eec-b7b5-8ab06c8ebe21 admin admin] PATH_INFO: `/v3/projects/8850424d4c4945dfb02012c49ca545df/users/fc355431025e4cdaa165de0d67e4ef61/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:56.599148 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24395765-2aaa-4eec-b7b5-8ab06c8ebe21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8850424d4c4945dfb02012c49ca545df, user_id=fc355431025e4cdaa165de0d67e4ef61, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:56.601878 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24395765-2aaa-4eec-b7b5-8ab06c8ebe21 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:56.612678 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1418/2823] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:56 2022] PUT /identity/v3/projects/8850424d4c4945dfb02012c49ca545df/users/fc355431025e4cdaa165de0d67e4ef61/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:56.622762 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f2c19-b54a-4c04-99ff-09d534e360bd None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:56.623269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f2c19-b54a-4c04-99ff-09d534e360bd None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:56.623371 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f2c19-b54a-4c04-99ff-09d534e360bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:56.649837 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-579f2c19-b54a-4c04-99ff-09d534e360bd None None] MFA Rules not processed for user `fc355431025e4cdaa165de0d67e4ef61`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:56.651045 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-579f2c19-b54a-4c04-99ff-09d534e360bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:56.715403 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:56.720096 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1406/2824] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:47:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:56.731691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24395765-2aaa-4eec-b7b5-8ab06c8ebe21 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:56.733409 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-24395765-2aaa-4eec-b7b5-8ab06c8ebe21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:56.789741 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24395765-2aaa-4eec-b7b5-8ab06c8ebe21 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:56.793607 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2c53e-3b2b-4d87-8b47-2209e2a939ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:56.794243 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2c53e-3b2b-4d87-8b47-2209e2a939ab service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:56.794529 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2c53e-3b2b-4d87-8b47-2209e2a939ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:56.794808 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2c53e-3b2b-4d87-8b47-2209e2a939ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:56.804240 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2c53e-3b2b-4d87-8b47-2209e2a939ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:56.805659 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2c53e-3b2b-4d87-8b47-2209e2a939ab service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:56.815512 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1419/2825] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:57.114520 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f2c19-b54a-4c04-99ff-09d534e360bd None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:57.124145 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f2c19-b54a-4c04-99ff-09d534e360bd None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:57.131713 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0156f-c464-4534-a4c4-131404206cb4 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc355431025e4cdaa165de0d67e4ef61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8850424d4c4945dfb02012c49ca545df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:57.132466 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0156f-c464-4534-a4c4-131404206cb4 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:57.132792 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0156f-c464-4534-a4c4-131404206cb4 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:57.133051 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0156f-c464-4534-a4c4-131404206cb4 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:57.133798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0156f-c464-4534-a4c4-131404206cb4 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:47:57.134117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0156f-c464-4534-a4c4-131404206cb4 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:57.135796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0156f-c464-4534-a4c4-131404206cb4 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:57.148642 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1407/2826] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:47:57.152873 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2c53e-3b2b-4d87-8b47-2209e2a939ab service cinder] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:57.162096 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2c53e-3b2b-4d87-8b47-2209e2a939ab service cinder] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:57.166691 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49736fa1-1019-47f7-aa2c-dca25d96a1d5 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc355431025e4cdaa165de0d67e4ef61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8850424d4c4945dfb02012c49ca545df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:57.167318 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49736fa1-1019-47f7-aa2c-dca25d96a1d5 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:57.167606 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49736fa1-1019-47f7-aa2c-dca25d96a1d5 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:57.167891 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49736fa1-1019-47f7-aa2c-dca25d96a1d5 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:57.168841 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49736fa1-1019-47f7-aa2c-dca25d96a1d5 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:57.170721 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49736fa1-1019-47f7-aa2c-dca25d96a1d5 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:57.199374 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1420/2827] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:57 2022] POST /identity/v3/projects => generated 354 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:47:57.203647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0156f-c464-4534-a4c4-131404206cb4 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:57.213712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0156f-c464-4534-a4c4-131404206cb4 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:57.270587 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d78163-bc4a-4f23-8c4e-9aefd2ada248 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc355431025e4cdaa165de0d67e4ef61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8850424d4c4945dfb02012c49ca545df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:57.271291 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d78163-bc4a-4f23-8c4e-9aefd2ada248 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:57.271589 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d78163-bc4a-4f23-8c4e-9aefd2ada248 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:57.271877 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d78163-bc4a-4f23-8c4e-9aefd2ada248 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:57.272607 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d78163-bc4a-4f23-8c4e-9aefd2ada248 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:47:57.272919 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d78163-bc4a-4f23-8c4e-9aefd2ada248 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:57.274511 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d78163-bc4a-4f23-8c4e-9aefd2ada248 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:57.286258 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1408/2828] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:47:57.324952 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49736fa1-1019-47f7-aa2c-dca25d96a1d5 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:57.336066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49736fa1-1019-47f7-aa2c-dca25d96a1d5 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:57.340791 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc9216-cbd6-465c-9586-19f448870510 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc355431025e4cdaa165de0d67e4ef61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8850424d4c4945dfb02012c49ca545df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:57.341700 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc9216-cbd6-465c-9586-19f448870510 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:57.342017 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc9216-cbd6-465c-9586-19f448870510 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:57.342304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc9216-cbd6-465c-9586-19f448870510 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] PATH_INFO: `/v3/projects/2d6a32115e604f01b6d388bb0e5cdd58` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:57.343528 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc9216-cbd6-465c-9586-19f448870510 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2d6a32115e604f01b6d388bb0e5cdd58)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:57.345435 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc9216-cbd6-465c-9586-19f448870510 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:57.367778 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:57.392234 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7acc9216-cbd6-465c-9586-19f448870510 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d6a32115e604f01b6d388bb0e5cdd58', 'request_id': 'req-7acc9216-cbd6-465c-9586-19f448870510'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:57.398514 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7acc9216-cbd6-465c-9586-19f448870510 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d6a32115e604f01b6d388bb0e5cdd58', 'request_id': 'req-7acc9216-cbd6-465c-9586-19f448870510'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:57.406150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7acc9216-cbd6-465c-9586-19f448870510 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d6a32115e604f01b6d388bb0e5cdd58', 'request_id': 'req-7acc9216-cbd6-465c-9586-19f448870510'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:57.416370 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7acc9216-cbd6-465c-9586-19f448870510 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] The token cache is being invalidate because project 2d6a32115e604f01b6d388bb0e5cdd58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:57.416667 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7acc9216-cbd6-465c-9586-19f448870510 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:57.417382 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-7acc9216-cbd6-465c-9586-19f448870510 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:57.419160 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1421/2829] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:57 2022] DELETE /identity/v3/projects/2d6a32115e604f01b6d388bb0e5cdd58 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:57.608300 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d78163-bc4a-4f23-8c4e-9aefd2ada248 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:57.610040 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-d4d78163-bc4a-4f23-8c4e-9aefd2ada248 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:57.674639 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d78163-bc4a-4f23-8c4e-9aefd2ada248 tempest-VolumeQuotaClassesTest-1193184356 tempest-VolumeQuotaClassesTest-1193184356-project-admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:57.729264 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2adc8-3a0c-4316-a76b-f9d2393cd717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:57.730129 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2adc8-3a0c-4316-a76b-f9d2393cd717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:57.730412 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2adc8-3a0c-4316-a76b-f9d2393cd717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:57.730692 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2adc8-3a0c-4316-a76b-f9d2393cd717 admin admin] PATH_INFO: `/v3/users/3d0b100b4d3d4758a0375978e2a01a7e` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:57.731898 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2adc8-3a0c-4316-a76b-f9d2393cd717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d0b100b4d3d4758a0375978e2a01a7e)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:57.733787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2adc8-3a0c-4316-a76b-f9d2393cd717 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:57.764985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:57.779296 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cab2adc8-3a0c-4316-a76b-f9d2393cd717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d0b100b4d3d4758a0375978e2a01a7e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:57.786659 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cab2adc8-3a0c-4316-a76b-f9d2393cd717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d0b100b4d3d4758a0375978e2a01a7e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:57.794187 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cab2adc8-3a0c-4316-a76b-f9d2393cd717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d0b100b4d3d4758a0375978e2a01a7e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:57.794808 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cab2adc8-3a0c-4316-a76b-f9d2393cd717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d0b100b4d3d4758a0375978e2a01a7e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:57.795436 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-cab2adc8-3a0c-4316-a76b-f9d2393cd717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d0b100b4d3d4758a0375978e2a01a7e'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:57.812729 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1409/2830] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:57 2022] DELETE /identity/v3/users/3d0b100b4d3d4758a0375978e2a01a7e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:57.817401 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8505736-4ae6-4aa9-933e-3fb5591ee922 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:57.817898 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8505736-4ae6-4aa9-933e-3fb5591ee922 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:57.818010 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8505736-4ae6-4aa9-933e-3fb5591ee922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:57.835955 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-d8505736-4ae6-4aa9-933e-3fb5591ee922 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:57.837166 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d8505736-4ae6-4aa9-933e-3fb5591ee922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:57.895331 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1422/2831] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:47:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:57.904842 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2adc8-3a0c-4316-a76b-f9d2393cd717 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:57.906242 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-cab2adc8-3a0c-4316-a76b-f9d2393cd717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:57.962323 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2adc8-3a0c-4316-a76b-f9d2393cd717 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:57.965866 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0676990-f4d7-42ec-918c-5bd808c3cf43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:57.966505 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0676990-f4d7-42ec-918c-5bd808c3cf43 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:57.966786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0676990-f4d7-42ec-918c-5bd808c3cf43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:57.967095 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0676990-f4d7-42ec-918c-5bd808c3cf43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:57.978197 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0676990-f4d7-42ec-918c-5bd808c3cf43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:57.979699 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0676990-f4d7-42ec-918c-5bd808c3cf43 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:57.989270 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1410/2832] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:47:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:47:58.255570 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8505736-4ae6-4aa9-933e-3fb5591ee922 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:58.257035 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-d8505736-4ae6-4aa9-933e-3fb5591ee922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:58.271578 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8505736-4ae6-4aa9-933e-3fb5591ee922 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:58.276128 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa795d65-c8f2-4485-8670-31c94c386944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:58.276992 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa795d65-c8f2-4485-8670-31c94c386944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:58.277278 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa795d65-c8f2-4485-8670-31c94c386944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:58.277586 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa795d65-c8f2-4485-8670-31c94c386944 admin admin] PATH_INFO: `/v3/projects/04d77659ad9d429bab8449440909d4e5` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:58.278762 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa795d65-c8f2-4485-8670-31c94c386944 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04d77659ad9d429bab8449440909d4e5)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:58.280711 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa795d65-c8f2-4485-8670-31c94c386944 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:58.327594 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-fa795d65-c8f2-4485-8670-31c94c386944 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04d77659ad9d429bab8449440909d4e5', 'request_id': 'req-fa795d65-c8f2-4485-8670-31c94c386944'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:58.334389 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-fa795d65-c8f2-4485-8670-31c94c386944 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04d77659ad9d429bab8449440909d4e5', 'request_id': 'req-fa795d65-c8f2-4485-8670-31c94c386944'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:58.341480 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-fa795d65-c8f2-4485-8670-31c94c386944 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04d77659ad9d429bab8449440909d4e5', 'request_id': 'req-fa795d65-c8f2-4485-8670-31c94c386944'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:58.351746 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-fa795d65-c8f2-4485-8670-31c94c386944 admin admin] The token cache is being invalidate because project 04d77659ad9d429bab8449440909d4e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:58.352036 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-fa795d65-c8f2-4485-8670-31c94c386944 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:58.352651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-fa795d65-c8f2-4485-8670-31c94c386944 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:58.354148 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1423/2833] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:58 2022] DELETE /identity/v3/projects/04d77659ad9d429bab8449440909d4e5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:58.358221 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0676990-f4d7-42ec-918c-5bd808c3cf43 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:58.369062 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0676990-f4d7-42ec-918c-5bd808c3cf43 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:58.427084 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f689069-d635-46ca-8e43-99631a297ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:58.427955 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f689069-d635-46ca-8e43-99631a297ba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:58.428243 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f689069-d635-46ca-8e43-99631a297ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:58.428526 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f689069-d635-46ca-8e43-99631a297ba4 admin admin] PATH_INFO: `/v3/users/fc355431025e4cdaa165de0d67e4ef61` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:58.429726 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f689069-d635-46ca-8e43-99631a297ba4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc355431025e4cdaa165de0d67e4ef61)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:58.431674 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f689069-d635-46ca-8e43-99631a297ba4 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:58.479347 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f689069-d635-46ca-8e43-99631a297ba4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc355431025e4cdaa165de0d67e4ef61'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:58.483832 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:58.487473 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f689069-d635-46ca-8e43-99631a297ba4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc355431025e4cdaa165de0d67e4ef61'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:58.495982 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f689069-d635-46ca-8e43-99631a297ba4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc355431025e4cdaa165de0d67e4ef61'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:58.496858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f689069-d635-46ca-8e43-99631a297ba4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc355431025e4cdaa165de0d67e4ef61'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:58.497463 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f689069-d635-46ca-8e43-99631a297ba4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc355431025e4cdaa165de0d67e4ef61'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:58.515057 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1411/2834] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:47:58 2022] DELETE /identity/v3/users/fc355431025e4cdaa165de0d67e4ef61 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:58.786820 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa795d65-c8f2-4485-8670-31c94c386944 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:58.788497 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-fa795d65-c8f2-4485-8670-31c94c386944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:58.803388 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa795d65-c8f2-4485-8670-31c94c386944 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:58.834902 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:58.863013 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f90d7-e1c9-46df-b108-be6249c02af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:58.863933 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f90d7-e1c9-46df-b108-be6249c02af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:58.864333 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f90d7-e1c9-46df-b108-be6249c02af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:58.864732 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f90d7-e1c9-46df-b108-be6249c02af0 admin admin] PATH_INFO: `/v3/projects/8850424d4c4945dfb02012c49ca545df` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:58.866096 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f90d7-e1c9-46df-b108-be6249c02af0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8850424d4c4945dfb02012c49ca545df)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:58.868162 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f90d7-e1c9-46df-b108-be6249c02af0 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:58.913072 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-2d4f90d7-e1c9-46df-b108-be6249c02af0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8850424d4c4945dfb02012c49ca545df', 'request_id': 'req-2d4f90d7-e1c9-46df-b108-be6249c02af0'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:58.919247 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-2d4f90d7-e1c9-46df-b108-be6249c02af0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8850424d4c4945dfb02012c49ca545df', 'request_id': 'req-2d4f90d7-e1c9-46df-b108-be6249c02af0'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:58.926400 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-2d4f90d7-e1c9-46df-b108-be6249c02af0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8850424d4c4945dfb02012c49ca545df', 'request_id': 'req-2d4f90d7-e1c9-46df-b108-be6249c02af0'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:58.936483 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-2d4f90d7-e1c9-46df-b108-be6249c02af0 admin admin] The token cache is being invalidate because project 8850424d4c4945dfb02012c49ca545df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:47:58.936771 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-2d4f90d7-e1c9-46df-b108-be6249c02af0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:58.937416 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-2d4f90d7-e1c9-46df-b108-be6249c02af0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:47:58.939477 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1424/2835] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:47:58 2022] DELETE /identity/v3/projects/8850424d4c4945dfb02012c49ca545df => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:58.944433 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1b05d-e49c-4544-9a33-020d2e8d2c64 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:58.944902 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1b05d-e49c-4544-9a33-020d2e8d2c64 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:58.945001 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1b05d-e49c-4544-9a33-020d2e8d2c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:58.963459 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-7ef1b05d-e49c-4544-9a33-020d2e8d2c64 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:58.964704 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-7ef1b05d-e49c-4544-9a33-020d2e8d2c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:59.024199 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1412/2836] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:47:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:59.033262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d04d5-2841-427d-810d-dcaa3ab1be4e None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:59.033495 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d04d5-2841-427d-810d-dcaa3ab1be4e None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:59.033774 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d04d5-2841-427d-810d-dcaa3ab1be4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:59.055116 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-3a1d04d5-2841-427d-810d-dcaa3ab1be4e None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:59.056262 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-3a1d04d5-2841-427d-810d-dcaa3ab1be4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:59.064817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1425/2837] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:47:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:59.069050 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1b05d-e49c-4544-9a33-020d2e8d2c64 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:59.079532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1b05d-e49c-4544-9a33-020d2e8d2c64 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:59.084220 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56603e2e-b4f2-45ea-a980-04c0b89cd637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:59.084879 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56603e2e-b4f2-45ea-a980-04c0b89cd637 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:59.085170 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56603e2e-b4f2-45ea-a980-04c0b89cd637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:59.085453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56603e2e-b4f2-45ea-a980-04c0b89cd637 admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:59.086192 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56603e2e-b4f2-45ea-a980-04c0b89cd637 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:47:59.086508 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56603e2e-b4f2-45ea-a980-04c0b89cd637 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:59.088148 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56603e2e-b4f2-45ea-a980-04c0b89cd637 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:59.099814 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1413/2838] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:47:59.103862 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d04d5-2841-427d-810d-dcaa3ab1be4e None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:59.113488 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d04d5-2841-427d-810d-dcaa3ab1be4e None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:59.118054 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6419492-b568-44a7-b17f-1d6a9555dfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:59.118699 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6419492-b568-44a7-b17f-1d6a9555dfa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:59.119016 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6419492-b568-44a7-b17f-1d6a9555dfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:59.119296 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6419492-b568-44a7-b17f-1d6a9555dfa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:59.119973 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6419492-b568-44a7-b17f-1d6a9555dfa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:59.121856 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6419492-b568-44a7-b17f-1d6a9555dfa7 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:59.158674 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1426/2839] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:59 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:47:59.159942 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56603e2e-b4f2-45ea-a980-04c0b89cd637 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:59.170136 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56603e2e-b4f2-45ea-a980-04c0b89cd637 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:59.232242 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086a19f-ad28-4fea-8bc8-ad85a7619348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:59.232911 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086a19f-ad28-4fea-8bc8-ad85a7619348 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:59.233196 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086a19f-ad28-4fea-8bc8-ad85a7619348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:59.233475 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086a19f-ad28-4fea-8bc8-ad85a7619348 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:59.234153 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086a19f-ad28-4fea-8bc8-ad85a7619348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:59.236723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086a19f-ad28-4fea-8bc8-ad85a7619348 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:59.256586 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1414/2840] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:59 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:59.260956 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6419492-b568-44a7-b17f-1d6a9555dfa7 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:59.271588 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6419492-b568-44a7-b17f-1d6a9555dfa7 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:59.276188 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099b854-7e4e-46b4-9c84-bc805e020d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:59.276853 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099b854-7e4e-46b4-9c84-bc805e020d43 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:59.277140 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099b854-7e4e-46b4-9c84-bc805e020d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:59.277421 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099b854-7e4e-46b4-9c84-bc805e020d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:59.278153 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099b854-7e4e-46b4-9c84-bc805e020d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:59.279784 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099b854-7e4e-46b4-9c84-bc805e020d43 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:59.290910 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1427/2841] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:59.295068 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086a19f-ad28-4fea-8bc8-ad85a7619348 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:59.305508 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086a19f-ad28-4fea-8bc8-ad85a7619348 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:59.310312 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edef77a-25aa-49f1-b4f7-2d31cb53d7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:59.310989 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edef77a-25aa-49f1-b4f7-2d31cb53d7bc admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:59.311279 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edef77a-25aa-49f1-b4f7-2d31cb53d7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:59.311563 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edef77a-25aa-49f1-b4f7-2d31cb53d7bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:59.312316 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edef77a-25aa-49f1-b4f7-2d31cb53d7bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:59.314879 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edef77a-25aa-49f1-b4f7-2d31cb53d7bc admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:59.324838 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1415/2842] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:59.328683 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099b854-7e4e-46b4-9c84-bc805e020d43 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:59.338838 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099b854-7e4e-46b4-9c84-bc805e020d43 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:59.343521 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fc472-3f89-46c2-a5cf-1abe39bf04df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:59.344675 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fc472-3f89-46c2-a5cf-1abe39bf04df admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:59.345200 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fc472-3f89-46c2-a5cf-1abe39bf04df admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:59.345563 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fc472-3f89-46c2-a5cf-1abe39bf04df admin admin] PATH_INFO: `/v3/projects/2262428a26d847749695b897e55cd79b/users/ac88fd3749a44bd9ac97cea402cca7a0/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:59.364039 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fc472-3f89-46c2-a5cf-1abe39bf04df admin admin] RBAC: Authorizing `identity:create_grant(project_id=2262428a26d847749695b897e55cd79b, user_id=ac88fd3749a44bd9ac97cea402cca7a0, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:59.366719 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fc472-3f89-46c2-a5cf-1abe39bf04df admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:59.380233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1428/2843] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:59 2022] PUT /identity/v3/projects/2262428a26d847749695b897e55cd79b/users/ac88fd3749a44bd9ac97cea402cca7a0/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:47:59.390721 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3e653-ca3f-4507-9e74-a8f4ac0f0fb3 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:59.391085 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3e653-ca3f-4507-9e74-a8f4ac0f0fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:59.392025 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3e653-ca3f-4507-9e74-a8f4ac0f0fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:59.420203 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-80f3e653-ca3f-4507-9e74-a8f4ac0f0fb3 None None] MFA Rules not processed for user `ac88fd3749a44bd9ac97cea402cca7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:59.421418 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-80f3e653-ca3f-4507-9e74-a8f4ac0f0fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:59.444168 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:59.487278 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1416/2844] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:47:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:47:59.491709 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fc472-3f89-46c2-a5cf-1abe39bf04df admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:59.552998 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fc472-3f89-46c2-a5cf-1abe39bf04df admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:59.557610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6e4df-cd1a-41cf-b29d-e1cbbb2805a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:59.558271 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6e4df-cd1a-41cf-b29d-e1cbbb2805a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:59.558561 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6e4df-cd1a-41cf-b29d-e1cbbb2805a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:59.558842 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6e4df-cd1a-41cf-b29d-e1cbbb2805a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:59.559576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6e4df-cd1a-41cf-b29d-e1cbbb2805a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:59.561469 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6e4df-cd1a-41cf-b29d-e1cbbb2805a5 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:59.565227 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:47:59.588975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1429/2845] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:47:59 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:59.592923 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3e653-ca3f-4507-9e74-a8f4ac0f0fb3 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:59.603513 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3e653-ca3f-4507-9e74-a8f4ac0f0fb3 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:59.660835 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d77405-0763-46ed-9adb-bebc5ff9a830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:59.661503 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d77405-0763-46ed-9adb-bebc5ff9a830 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:59.661786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d77405-0763-46ed-9adb-bebc5ff9a830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:59.662069 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d77405-0763-46ed-9adb-bebc5ff9a830 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:59.662767 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d77405-0763-46ed-9adb-bebc5ff9a830 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:59.664783 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d77405-0763-46ed-9adb-bebc5ff9a830 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:59.685212 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1417/2846] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:47:59 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:47:59.690145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6e4df-cd1a-41cf-b29d-e1cbbb2805a5 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:59.700341 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6e4df-cd1a-41cf-b29d-e1cbbb2805a5 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:59.705665 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064b3b9-3f78-4dcc-b081-83a7524a44a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:59.706326 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064b3b9-3f78-4dcc-b081-83a7524a44a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:59.706610 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064b3b9-3f78-4dcc-b081-83a7524a44a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:59.706890 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064b3b9-3f78-4dcc-b081-83a7524a44a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:59.707644 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064b3b9-3f78-4dcc-b081-83a7524a44a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:59.709414 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064b3b9-3f78-4dcc-b081-83a7524a44a6 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:59.720695 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1430/2847] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:59.724845 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d77405-0763-46ed-9adb-bebc5ff9a830 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:59.734540 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d77405-0763-46ed-9adb-bebc5ff9a830 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:59.739762 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aca9c7-400e-4b5e-9954-1c6ac010365e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:59.740650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aca9c7-400e-4b5e-9954-1c6ac010365e admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:59.740919 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aca9c7-400e-4b5e-9954-1c6ac010365e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:59.741201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aca9c7-400e-4b5e-9954-1c6ac010365e admin admin] PATH_INFO: `/v3/projects/498356379dbb4afa8160c33ba442aaf6/users/e5f74bdbe7ac4d69b6e54b0d8c00db62/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:59.755304 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aca9c7-400e-4b5e-9954-1c6ac010365e admin admin] RBAC: Authorizing `identity:create_grant(project_id=498356379dbb4afa8160c33ba442aaf6, user_id=e5f74bdbe7ac4d69b6e54b0d8c00db62, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:59.757914 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aca9c7-400e-4b5e-9954-1c6ac010365e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:59.770814 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1418/2848] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:59 2022] PUT /identity/v3/projects/498356379dbb4afa8160c33ba442aaf6/users/e5f74bdbe7ac4d69b6e54b0d8c00db62/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:59.774684 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064b3b9-3f78-4dcc-b081-83a7524a44a6 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:59.785518 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064b3b9-3f78-4dcc-b081-83a7524a44a6 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:59.842284 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a99c0-218a-47c5-a4e8-3d724405df2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:59.842908 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a99c0-218a-47c5-a4e8-3d724405df2f admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:59.843183 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a99c0-218a-47c5-a4e8-3d724405df2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:59.843464 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a99c0-218a-47c5-a4e8-3d724405df2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:59.844176 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a99c0-218a-47c5-a4e8-3d724405df2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:59.846446 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a99c0-218a-47c5-a4e8-3d724405df2f admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:59.855981 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1431/2849] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:47:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:47:59.860957 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aca9c7-400e-4b5e-9954-1c6ac010365e admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:47:59.871324 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aca9c7-400e-4b5e-9954-1c6ac010365e admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:47:59.876613 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592d933-a0f8-4f15-8d87-f1dcfb35e667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:47:59.877601 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592d933-a0f8-4f15-8d87-f1dcfb35e667 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:59.877888 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592d933-a0f8-4f15-8d87-f1dcfb35e667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:59.878168 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592d933-a0f8-4f15-8d87-f1dcfb35e667 admin admin] PATH_INFO: `/v3/projects/498356379dbb4afa8160c33ba442aaf6/users/e5f74bdbe7ac4d69b6e54b0d8c00db62/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:59.880129 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592d933-a0f8-4f15-8d87-f1dcfb35e667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=498356379dbb4afa8160c33ba442aaf6, user_id=e5f74bdbe7ac4d69b6e54b0d8c00db62, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:47:59.882755 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592d933-a0f8-4f15-8d87-f1dcfb35e667 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:47:59.893647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1419/2850] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:47:59 2022] PUT /identity/v3/projects/498356379dbb4afa8160c33ba442aaf6/users/e5f74bdbe7ac4d69b6e54b0d8c00db62/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:47:59.903823 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c865e85-1ae2-4f07-a5fc-5066afa8f0b1 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:47:59.904118 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c865e85-1ae2-4f07-a5fc-5066afa8f0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:47:59.904398 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c865e85-1ae2-4f07-a5fc-5066afa8f0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:47:59.930767 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-6c865e85-1ae2-4f07-a5fc-5066afa8f0b1 None None] MFA Rules not processed for user `e5f74bdbe7ac4d69b6e54b0d8c00db62`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:47:59.932118 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-6c865e85-1ae2-4f07-a5fc-5066afa8f0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:47:59.999554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1432/2851] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:47:59 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:00.011505 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592d933-a0f8-4f15-8d87-f1dcfb35e667 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:00.013225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-5592d933-a0f8-4f15-8d87-f1dcfb35e667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:00.070431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592d933-a0f8-4f15-8d87-f1dcfb35e667 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:00.074042 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8d6f7-1873-4d0f-aa1a-0528f65b5500 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:00.074699 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8d6f7-1873-4d0f-aa1a-0528f65b5500 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:00.075011 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8d6f7-1873-4d0f-aa1a-0528f65b5500 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:00.075297 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8d6f7-1873-4d0f-aa1a-0528f65b5500 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:00.084359 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8d6f7-1873-4d0f-aa1a-0528f65b5500 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:00.085939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8d6f7-1873-4d0f-aa1a-0528f65b5500 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:00.095326 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1420/2852] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:00 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:48:00.174158 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c865e85-1ae2-4f07-a5fc-5066afa8f0b1 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:00.183939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c865e85-1ae2-4f07-a5fc-5066afa8f0b1 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:00.187575 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716b368-1679-47ec-a52d-d54dc2d9d349 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:00.188815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716b368-1679-47ec-a52d-d54dc2d9d349 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:00.189106 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716b368-1679-47ec-a52d-d54dc2d9d349 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:00.189392 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716b368-1679-47ec-a52d-d54dc2d9d349 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:00.241459 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:00.246347 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716b368-1679-47ec-a52d-d54dc2d9d349 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:00.248152 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716b368-1679-47ec-a52d-d54dc2d9d349 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:00.257865 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1433/2853] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:00 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:48:04.941864 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8d6f7-1873-4d0f-aa1a-0528f65b5500 service cinder] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:04.968782 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:05.024309 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8d6f7-1873-4d0f-aa1a-0528f65b5500 service cinder] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:05.030026 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8a053-830f-49ed-8c52-44c6e4593e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:05.030910 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8a053-830f-49ed-8c52-44c6e4593e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:05.031226 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8a053-830f-49ed-8c52-44c6e4593e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:05.031510 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8a053-830f-49ed-8c52-44c6e4593e5c admin admin] PATH_INFO: `/v3/users/ac88fd3749a44bd9ac97cea402cca7a0` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:05.033216 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8a053-830f-49ed-8c52-44c6e4593e5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac88fd3749a44bd9ac97cea402cca7a0)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:05.035162 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8a053-830f-49ed-8c52-44c6e4593e5c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:05.093645 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-88b8a053-830f-49ed-8c52-44c6e4593e5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac88fd3749a44bd9ac97cea402cca7a0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:05.102196 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-88b8a053-830f-49ed-8c52-44c6e4593e5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac88fd3749a44bd9ac97cea402cca7a0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:05.114768 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-88b8a053-830f-49ed-8c52-44c6e4593e5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac88fd3749a44bd9ac97cea402cca7a0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:05.115329 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-88b8a053-830f-49ed-8c52-44c6e4593e5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac88fd3749a44bd9ac97cea402cca7a0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:05.115950 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-88b8a053-830f-49ed-8c52-44c6e4593e5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac88fd3749a44bd9ac97cea402cca7a0'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:05.138831 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1421/2854] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:48:04 2022] DELETE /identity/v3/users/ac88fd3749a44bd9ac97cea402cca7a0 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:05.140600 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80b4d0-d001-4aa8-80fd-ccabfe0cb673 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:05.141135 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80b4d0-d001-4aa8-80fd-ccabfe0cb673 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:05.142530 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80b4d0-d001-4aa8-80fd-ccabfe0cb673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:05.164871 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-ab80b4d0-d001-4aa8-80fd-ccabfe0cb673 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:05.166110 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-ab80b4d0-d001-4aa8-80fd-ccabfe0cb673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:05.237688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1434/2855] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:48:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:48:05.248355 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8a053-830f-49ed-8c52-44c6e4593e5c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:05.249908 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-88b8a053-830f-49ed-8c52-44c6e4593e5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:05.331843 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8a053-830f-49ed-8c52-44c6e4593e5c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:05.335502 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a179655-857d-4f66-b4be-055f5e413d65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:05.336151 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a179655-857d-4f66-b4be-055f5e413d65 service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:05.336436 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a179655-857d-4f66-b4be-055f5e413d65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:05.336717 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a179655-857d-4f66-b4be-055f5e413d65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:05.347769 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a179655-857d-4f66-b4be-055f5e413d65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:05.349188 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a179655-857d-4f66-b4be-055f5e413d65 service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:05.359350 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1422/2856] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 22:48:05.643109 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80b4d0-d001-4aa8-80fd-ccabfe0cb673 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:05.644544 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-ab80b4d0-d001-4aa8-80fd-ccabfe0cb673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:05.659980 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80b4d0-d001-4aa8-80fd-ccabfe0cb673 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:05.664635 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1eb0b-bcdf-40df-8c36-d9b288b053ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:05.666263 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1eb0b-bcdf-40df-8c36-d9b288b053ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:05.666552 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1eb0b-bcdf-40df-8c36-d9b288b053ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:05.666886 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1eb0b-bcdf-40df-8c36-d9b288b053ec admin admin] PATH_INFO: `/v3/projects/2262428a26d847749695b897e55cd79b` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:05.673280 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1eb0b-bcdf-40df-8c36-d9b288b053ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=2262428a26d847749695b897e55cd79b)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:05.675269 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f1eb0b-bcdf-40df-8c36-d9b288b053ec admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:05.722531 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-68f1eb0b-bcdf-40df-8c36-d9b288b053ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2262428a26d847749695b897e55cd79b', 'request_id': 'req-68f1eb0b-bcdf-40df-8c36-d9b288b053ec'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:05.728817 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-68f1eb0b-bcdf-40df-8c36-d9b288b053ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2262428a26d847749695b897e55cd79b', 'request_id': 'req-68f1eb0b-bcdf-40df-8c36-d9b288b053ec'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:05.736725 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-68f1eb0b-bcdf-40df-8c36-d9b288b053ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2262428a26d847749695b897e55cd79b', 'request_id': 'req-68f1eb0b-bcdf-40df-8c36-d9b288b053ec'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:05.746850 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-68f1eb0b-bcdf-40df-8c36-d9b288b053ec admin admin] The token cache is being invalidate because project 2262428a26d847749695b897e55cd79b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:48:05.747170 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-68f1eb0b-bcdf-40df-8c36-d9b288b053ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:05.747813 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-68f1eb0b-bcdf-40df-8c36-d9b288b053ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:05.749348 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1435/2857] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:48:05 2022] DELETE /identity/v3/projects/2262428a26d847749695b897e55cd79b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:05.753339 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a179655-857d-4f66-b4be-055f5e413d65 service neutron] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:05.762741 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a179655-857d-4f66-b4be-055f5e413d65 service neutron] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:05.803489 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:05.821219 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e3b6d-d7ee-49bf-a1e4-791ef1aad53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:05.822083 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e3b6d-d7ee-49bf-a1e4-791ef1aad53e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:05.822369 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e3b6d-d7ee-49bf-a1e4-791ef1aad53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:05.822647 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e3b6d-d7ee-49bf-a1e4-791ef1aad53e admin admin] PATH_INFO: `/v3/users/e5f74bdbe7ac4d69b6e54b0d8c00db62` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:05.823848 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e3b6d-d7ee-49bf-a1e4-791ef1aad53e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5f74bdbe7ac4d69b6e54b0d8c00db62)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:05.825738 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e3b6d-d7ee-49bf-a1e4-791ef1aad53e admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:05.871064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f7e3b6d-d7ee-49bf-a1e4-791ef1aad53e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5f74bdbe7ac4d69b6e54b0d8c00db62'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:05.878490 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f7e3b6d-d7ee-49bf-a1e4-791ef1aad53e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5f74bdbe7ac4d69b6e54b0d8c00db62'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:05.885805 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f7e3b6d-d7ee-49bf-a1e4-791ef1aad53e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5f74bdbe7ac4d69b6e54b0d8c00db62'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:05.886375 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f7e3b6d-d7ee-49bf-a1e4-791ef1aad53e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5f74bdbe7ac4d69b6e54b0d8c00db62'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:05.887007 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-8f7e3b6d-d7ee-49bf-a1e4-791ef1aad53e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5f74bdbe7ac4d69b6e54b0d8c00db62'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:05.905064 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1423/2858] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:48:05 2022] DELETE /identity/v3/users/e5f74bdbe7ac4d69b6e54b0d8c00db62 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:06.174306 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1eb0b-bcdf-40df-8c36-d9b288b053ec admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:06.175824 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-68f1eb0b-bcdf-40df-8c36-d9b288b053ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:06.189228 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1eb0b-bcdf-40df-8c36-d9b288b053ec admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:06.246687 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbbb31-2612-4bc0-9aa3-f61bea3867d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:06.247592 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbbb31-2612-4bc0-9aa3-f61bea3867d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:06.247884 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbbb31-2612-4bc0-9aa3-f61bea3867d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:06.248170 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbbb31-2612-4bc0-9aa3-f61bea3867d0 admin admin] PATH_INFO: `/v3/projects/498356379dbb4afa8160c33ba442aaf6` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:06.249343 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbbb31-2612-4bc0-9aa3-f61bea3867d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=498356379dbb4afa8160c33ba442aaf6)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:06.251868 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbbb31-2612-4bc0-9aa3-f61bea3867d0 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:06.273050 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:06.298840 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-11fbbb31-2612-4bc0-9aa3-f61bea3867d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '498356379dbb4afa8160c33ba442aaf6', 'request_id': 'req-11fbbb31-2612-4bc0-9aa3-f61bea3867d0'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:06.304954 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-11fbbb31-2612-4bc0-9aa3-f61bea3867d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '498356379dbb4afa8160c33ba442aaf6', 'request_id': 'req-11fbbb31-2612-4bc0-9aa3-f61bea3867d0'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:06.311625 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-11fbbb31-2612-4bc0-9aa3-f61bea3867d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '498356379dbb4afa8160c33ba442aaf6', 'request_id': 'req-11fbbb31-2612-4bc0-9aa3-f61bea3867d0'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:06.322339 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-11fbbb31-2612-4bc0-9aa3-f61bea3867d0 admin admin] The token cache is being invalidate because project 498356379dbb4afa8160c33ba442aaf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:48:06.322633 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-11fbbb31-2612-4bc0-9aa3-f61bea3867d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:06.323285 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-11fbbb31-2612-4bc0-9aa3-f61bea3867d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:06.324828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1436/2859] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:48:06 2022] DELETE /identity/v3/projects/498356379dbb4afa8160c33ba442aaf6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:48:06.330430 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdade2a5-fa7c-4683-81cc-87a8316edd24 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:06.330913 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdade2a5-fa7c-4683-81cc-87a8316edd24 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:06.331115 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdade2a5-fa7c-4683-81cc-87a8316edd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:06.348795 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fdade2a5-fa7c-4683-81cc-87a8316edd24 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:06.349996 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fdade2a5-fa7c-4683-81cc-87a8316edd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:06.409582 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1424/2860] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:48:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:06.417529 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de19761-01fc-4aa8-b39e-1c71b5758347 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:06.417761 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de19761-01fc-4aa8-b39e-1c71b5758347 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:06.418047 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de19761-01fc-4aa8-b39e-1c71b5758347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:06.436159 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-9de19761-01fc-4aa8-b39e-1c71b5758347 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:06.437358 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-9de19761-01fc-4aa8-b39e-1c71b5758347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:06.445908 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1437/2861] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:48:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:06.450117 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdade2a5-fa7c-4683-81cc-87a8316edd24 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:06.460460 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdade2a5-fa7c-4683-81cc-87a8316edd24 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:06.465641 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5e400-5b9c-4173-809f-f95c9515294c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:06.466803 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5e400-5b9c-4173-809f-f95c9515294c admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:06.467303 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5e400-5b9c-4173-809f-f95c9515294c admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:06.467598 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5e400-5b9c-4173-809f-f95c9515294c admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:06.468370 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5e400-5b9c-4173-809f-f95c9515294c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:48:06.468691 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5e400-5b9c-4173-809f-f95c9515294c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:06.470361 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb5e400-5b9c-4173-809f-f95c9515294c admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:06.482122 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1425/2862] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 22:48:06.486602 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de19761-01fc-4aa8-b39e-1c71b5758347 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:06.495888 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de19761-01fc-4aa8-b39e-1c71b5758347 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:06.501174 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c7c64-07dd-434b-a939-58ca478d71d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:06.501833 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c7c64-07dd-434b-a939-58ca478d71d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:06.502118 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c7c64-07dd-434b-a939-58ca478d71d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:06.502402 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c7c64-07dd-434b-a939-58ca478d71d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:06.503131 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c7c64-07dd-434b-a939-58ca478d71d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:06.505028 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c7c64-07dd-434b-a939-58ca478d71d0 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:06.529941 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1438/2863] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:48:06 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:48:06.533830 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5e400-5b9c-4173-809f-f95c9515294c admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:06.544096 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb5e400-5b9c-4173-809f-f95c9515294c admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:06.588587 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:06.601544 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292cb65-1239-42f4-a397-3664b34118cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:06.602216 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292cb65-1239-42f4-a397-3664b34118cd admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:06.602505 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292cb65-1239-42f4-a397-3664b34118cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:06.602788 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292cb65-1239-42f4-a397-3664b34118cd admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:06.603540 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292cb65-1239-42f4-a397-3664b34118cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:06.605614 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292cb65-1239-42f4-a397-3664b34118cd admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:06.629491 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1426/2864] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:48:06 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:48:06.633673 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c7c64-07dd-434b-a939-58ca478d71d0 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:06.645238 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c7c64-07dd-434b-a939-58ca478d71d0 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:06.649857 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13905141-68a2-425f-891e-1990fcb05db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:06.650555 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13905141-68a2-425f-891e-1990fcb05db7 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:06.650842 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13905141-68a2-425f-891e-1990fcb05db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:06.651169 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13905141-68a2-425f-891e-1990fcb05db7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:06.651899 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13905141-68a2-425f-891e-1990fcb05db7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:06.653518 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13905141-68a2-425f-891e-1990fcb05db7 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:06.663774 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1439/2865] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:48:06.667625 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292cb65-1239-42f4-a397-3664b34118cd admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:06.678270 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292cb65-1239-42f4-a397-3664b34118cd admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:06.682933 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2abade-3c8f-4068-95d3-079b74cd0851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:06.683602 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2abade-3c8f-4068-95d3-079b74cd0851 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:06.683888 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2abade-3c8f-4068-95d3-079b74cd0851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:06.684173 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2abade-3c8f-4068-95d3-079b74cd0851 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:06.684908 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2abade-3c8f-4068-95d3-079b74cd0851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:06.686485 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2abade-3c8f-4068-95d3-079b74cd0851 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:06.700242 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1427/2866] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:48:06.704519 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13905141-68a2-425f-891e-1990fcb05db7 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:06.714330 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13905141-68a2-425f-891e-1990fcb05db7 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:06.719200 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679da02-b5a0-4dc9-aac9-189816f7ad98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:06.720226 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679da02-b5a0-4dc9-aac9-189816f7ad98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:06.720544 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679da02-b5a0-4dc9-aac9-189816f7ad98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:06.720829 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679da02-b5a0-4dc9-aac9-189816f7ad98 admin admin] PATH_INFO: `/v3/projects/26a554ac2c914ace9c25ceb0b345be26/users/9ed15be86477496a969ec48355af8ad1/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:06.735828 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0679da02-b5a0-4dc9-aac9-189816f7ad98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26a554ac2c914ace9c25ceb0b345be26, user_id=9ed15be86477496a969ec48355af8ad1, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:06.738517 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0679da02-b5a0-4dc9-aac9-189816f7ad98 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:06.755363 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1440/2867] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:48:06 2022] PUT /identity/v3/projects/26a554ac2c914ace9c25ceb0b345be26/users/9ed15be86477496a969ec48355af8ad1/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:06.770460 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cf072-6137-4334-b166-234d2dbdba23 None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:06.770753 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cf072-6137-4334-b166-234d2dbdba23 None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:06.771071 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cf072-6137-4334-b166-234d2dbdba23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:06.797939 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-1b2cf072-6137-4334-b166-234d2dbdba23 None None] MFA Rules not processed for user `9ed15be86477496a969ec48355af8ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:06.799202 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-1b2cf072-6137-4334-b166-234d2dbdba23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:06.861858 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1428/2868] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:48:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:06.872880 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679da02-b5a0-4dc9-aac9-189816f7ad98 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:06.874652 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-0679da02-b5a0-4dc9-aac9-189816f7ad98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:06.931522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679da02-b5a0-4dc9-aac9-189816f7ad98 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:06.935046 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c90d27-9f9e-4c67-bfa5-cf47e9023546 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:06.935693 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c90d27-9f9e-4c67-bfa5-cf47e9023546 service cinder] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:06.935983 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c90d27-9f9e-4c67-bfa5-cf47e9023546 service cinder] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:06.936265 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c90d27-9f9e-4c67-bfa5-cf47e9023546 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:06.945358 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c90d27-9f9e-4c67-bfa5-cf47e9023546 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:06.946782 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c90d27-9f9e-4c67-bfa5-cf47e9023546 service cinder] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:06.955532 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1441/2869] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:48:06.974975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2cf072-6137-4334-b166-234d2dbdba23 None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:07.037755 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2cf072-6137-4334-b166-234d2dbdba23 None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:07.042274 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f6b80-234c-4c8f-920b-7f280f5fb6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:07.043431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f6b80-234c-4c8f-920b-7f280f5fb6d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:07.043717 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f6b80-234c-4c8f-920b-7f280f5fb6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:07.044004 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f6b80-234c-4c8f-920b-7f280f5fb6d1 admin admin] PATH_INFO: `/v3/users/9ed15be86477496a969ec48355af8ad1` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:07.045201 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f6b80-234c-4c8f-920b-7f280f5fb6d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ed15be86477496a969ec48355af8ad1)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:07.047130 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f6b80-234c-4c8f-920b-7f280f5fb6d1 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:07.092827 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-117f6b80-234c-4c8f-920b-7f280f5fb6d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ed15be86477496a969ec48355af8ad1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:07.100804 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-117f6b80-234c-4c8f-920b-7f280f5fb6d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ed15be86477496a969ec48355af8ad1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:07.103369 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:07.109092 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-117f6b80-234c-4c8f-920b-7f280f5fb6d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ed15be86477496a969ec48355af8ad1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:07.110514 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-117f6b80-234c-4c8f-920b-7f280f5fb6d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ed15be86477496a969ec48355af8ad1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:07.111581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-117f6b80-234c-4c8f-920b-7f280f5fb6d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ed15be86477496a969ec48355af8ad1'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:07.132903 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1429/2870] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:48:06 2022] DELETE /identity/v3/users/9ed15be86477496a969ec48355af8ad1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:07.137413 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8502e3-5945-4917-917f-eb250aca6391 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:07.137711 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8502e3-5945-4917-917f-eb250aca6391 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:07.137995 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8502e3-5945-4917-917f-eb250aca6391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:07.154723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-eb8502e3-5945-4917-917f-eb250aca6391 None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:07.155968 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-eb8502e3-5945-4917-917f-eb250aca6391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:07.198242 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:07.217066 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1442/2871] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:07.227459 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f6b80-234c-4c8f-920b-7f280f5fb6d1 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:07.228869 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-117f6b80-234c-4c8f-920b-7f280f5fb6d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:07.286975 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f6b80-234c-4c8f-920b-7f280f5fb6d1 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:07.290538 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa49180-7b63-473c-9cd9-9bb7a90ebcde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:07.291219 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa49180-7b63-473c-9cd9-9bb7a90ebcde service neutron] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:07.291504 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa49180-7b63-473c-9cd9-9bb7a90ebcde service neutron] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:07.291785 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa49180-7b63-473c-9cd9-9bb7a90ebcde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:07.301709 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa49180-7b63-473c-9cd9-9bb7a90ebcde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:07.303161 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa49180-7b63-473c-9cd9-9bb7a90ebcde service neutron] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:07.312530 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1430/2872] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:48:07.577945 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8502e3-5945-4917-917f-eb250aca6391 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:07.579748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-eb8502e3-5945-4917-917f-eb250aca6391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:07.593404 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8502e3-5945-4917-917f-eb250aca6391 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:07.598779 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92ca08-fb16-44e6-bad2-254f7a2129bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:07.600233 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92ca08-fb16-44e6-bad2-254f7a2129bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:07.600530 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92ca08-fb16-44e6-bad2-254f7a2129bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:07.600810 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92ca08-fb16-44e6-bad2-254f7a2129bb admin admin] PATH_INFO: `/v3/projects/26a554ac2c914ace9c25ceb0b345be26` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:07.602189 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92ca08-fb16-44e6-bad2-254f7a2129bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=26a554ac2c914ace9c25ceb0b345be26)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:07.604138 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92ca08-fb16-44e6-bad2-254f7a2129bb admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:07.652239 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-3f92ca08-fb16-44e6-bad2-254f7a2129bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26a554ac2c914ace9c25ceb0b345be26', 'request_id': 'req-3f92ca08-fb16-44e6-bad2-254f7a2129bb'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:07.658751 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-3f92ca08-fb16-44e6-bad2-254f7a2129bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26a554ac2c914ace9c25ceb0b345be26', 'request_id': 'req-3f92ca08-fb16-44e6-bad2-254f7a2129bb'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:07.666537 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-3f92ca08-fb16-44e6-bad2-254f7a2129bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26a554ac2c914ace9c25ceb0b345be26', 'request_id': 'req-3f92ca08-fb16-44e6-bad2-254f7a2129bb'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:07.678286 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-3f92ca08-fb16-44e6-bad2-254f7a2129bb admin admin] The token cache is being invalidate because project 26a554ac2c914ace9c25ceb0b345be26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80201) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:48:07.678576 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-3f92ca08-fb16-44e6-bad2-254f7a2129bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:07.679227 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-3f92ca08-fb16-44e6-bad2-254f7a2129bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:07.680834 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1443/2873] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:48:07 2022] DELETE /identity/v3/projects/26a554ac2c914ace9c25ceb0b345be26 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:07.685781 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163aecc5-4a72-43e5-b34f-dd266bb1ef9f None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:07.686254 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163aecc5-4a72-43e5-b34f-dd266bb1ef9f None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:07.686360 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163aecc5-4a72-43e5-b34f-dd266bb1ef9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:07.705931 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-163aecc5-4a72-43e5-b34f-dd266bb1ef9f None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:07.707096 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-163aecc5-4a72-43e5-b34f-dd266bb1ef9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:07.772459 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1431/2874] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:07.781114 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d28ee-4447-4413-ad43-d02dad4d816e None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:07.781413 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d28ee-4447-4413-ad43-d02dad4d816e None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:07.782261 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d28ee-4447-4413-ad43-d02dad4d816e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:07.801161 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-602d28ee-4447-4413-ad43-d02dad4d816e None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:07.802368 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-602d28ee-4447-4413-ad43-d02dad4d816e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:07.810229 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1444/2875] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:07.815453 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163aecc5-4a72-43e5-b34f-dd266bb1ef9f None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:07.825000 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163aecc5-4a72-43e5-b34f-dd266bb1ef9f None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:07.830723 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3de43c-62c5-4f87-b417-68fbec68afcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:07.831411 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3de43c-62c5-4f87-b417-68fbec68afcb admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:07.831704 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3de43c-62c5-4f87-b417-68fbec68afcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:07.831988 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3de43c-62c5-4f87-b417-68fbec68afcb admin admin] PATH_INFO: `/v3/domains` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:07.832712 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3de43c-62c5-4f87-b417-68fbec68afcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80203) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 22:48:07.833023 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3de43c-62c5-4f87-b417-68fbec68afcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:07.834604 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3de43c-62c5-4f87-b417-68fbec68afcb admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:07.846461 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1432/2876] 10.222.0.43 () {58 vars in 1152 bytes} [Thu Jun 23 22:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 22:48:07.850771 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d28ee-4447-4413-ad43-d02dad4d816e None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:07.860241 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d28ee-4447-4413-ad43-d02dad4d816e None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:07.865099 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be90496-cd2b-4fa4-8674-999ee882ac85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:07.865751 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be90496-cd2b-4fa4-8674-999ee882ac85 admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:07.866038 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be90496-cd2b-4fa4-8674-999ee882ac85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:07.866318 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be90496-cd2b-4fa4-8674-999ee882ac85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:07.867117 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be90496-cd2b-4fa4-8674-999ee882ac85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:07.869006 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be90496-cd2b-4fa4-8674-999ee882ac85 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:07.895237 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1445/2877] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:48:07 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 22:48:07.899499 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3de43c-62c5-4f87-b417-68fbec68afcb admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:07.909472 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3de43c-62c5-4f87-b417-68fbec68afcb admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:07.965610 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9a041-9dce-4454-9494-4c65d507c854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:07.966198 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9a041-9dce-4454-9494-4c65d507c854 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:07.966492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9a041-9dce-4454-9494-4c65d507c854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:07.966776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9a041-9dce-4454-9494-4c65d507c854 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:07.967498 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9a041-9dce-4454-9494-4c65d507c854 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:07.969650 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9a041-9dce-4454-9494-4c65d507c854 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:07.979202 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:07.991443 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1433/2878] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:48:07 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:48:07.996069 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be90496-cd2b-4fa4-8674-999ee882ac85 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:08.007493 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be90496-cd2b-4fa4-8674-999ee882ac85 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:08.012150 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8f7d0-b205-4e59-8897-b76c7792bbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:08.012815 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8f7d0-b205-4e59-8897-b76c7792bbe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:08.013104 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8f7d0-b205-4e59-8897-b76c7792bbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:08.013389 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8f7d0-b205-4e59-8897-b76c7792bbe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:08.014149 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8f7d0-b205-4e59-8897-b76c7792bbe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:08.015793 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa8f7d0-b205-4e59-8897-b76c7792bbe2 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:08.026755 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1446/2879] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:48:08.030688 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9a041-9dce-4454-9494-4c65d507c854 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:08.040985 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9a041-9dce-4454-9494-4c65d507c854 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:08.045776 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bcbbeb-4dbd-4a50-abc3-5eba832ef706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:08.046431 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcbbeb-4dbd-4a50-abc3-5eba832ef706 admin admin] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:08.046717 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcbbeb-4dbd-4a50-abc3-5eba832ef706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:08.047034 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcbbeb-4dbd-4a50-abc3-5eba832ef706 admin admin] PATH_INFO: `/v3/roles` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:08.047876 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bcbbeb-4dbd-4a50-abc3-5eba832ef706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:08.049448 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bcbbeb-4dbd-4a50-abc3-5eba832ef706 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:08.060131 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1434/2880] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:48:08.065121 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8f7d0-b205-4e59-8897-b76c7792bbe2 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:08.075569 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8f7d0-b205-4e59-8897-b76c7792bbe2 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:08.081423 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea49e1-6d3f-4640-99b2-ce5c95c8ccf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:08.082358 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea49e1-6d3f-4640-99b2-ce5c95c8ccf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:08.082718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea49e1-6d3f-4640-99b2-ce5c95c8ccf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:08.083029 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea49e1-6d3f-4640-99b2-ce5c95c8ccf9 admin admin] PATH_INFO: `/v3/projects/41fa6d09534d477f8b69dd5852a23b82/users/abf3485a10d64a13acd8f6779f609079/roles/1baba2239751440eb25af0d0fd2d072d` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:08.097644 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea49e1-6d3f-4640-99b2-ce5c95c8ccf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41fa6d09534d477f8b69dd5852a23b82, user_id=abf3485a10d64a13acd8f6779f609079, role_id=1baba2239751440eb25af0d0fd2d072d)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:08.100304 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea49e1-6d3f-4640-99b2-ce5c95c8ccf9 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:08.113996 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1447/2881] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:48:08 2022] PUT /identity/v3/projects/41fa6d09534d477f8b69dd5852a23b82/users/abf3485a10d64a13acd8f6779f609079/roles/1baba2239751440eb25af0d0fd2d072d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:08.176662 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe335235-7260-4e69-9fa3-683735de35ce None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:08.177005 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe335235-7260-4e69-9fa3-683735de35ce None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:08.177248 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe335235-7260-4e69-9fa3-683735de35ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:08.205058 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-fe335235-7260-4e69-9fa3-683735de35ce None None] MFA Rules not processed for user `abf3485a10d64a13acd8f6779f609079`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:08.206269 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-fe335235-7260-4e69-9fa3-683735de35ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:08.273492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1435/2882] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:48:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 22:48:08.278120 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea49e1-6d3f-4640-99b2-ce5c95c8ccf9 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:08.340070 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea49e1-6d3f-4640-99b2-ce5c95c8ccf9 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:08.345034 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b951ebd-5e14-465f-bdb7-5b61109e0c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:08.345698 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b951ebd-5e14-465f-bdb7-5b61109e0c2f admin admin] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:08.345987 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b951ebd-5e14-465f-bdb7-5b61109e0c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:08.346267 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b951ebd-5e14-465f-bdb7-5b61109e0c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:08.346993 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b951ebd-5e14-465f-bdb7-5b61109e0c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:08.348885 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b951ebd-5e14-465f-bdb7-5b61109e0c2f admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:08.358522 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:08.377984 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1448/2883] 10.222.0.43 () {60 vars in 1152 bytes} [Thu Jun 23 22:48:08 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:48:08.382746 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe335235-7260-4e69-9fa3-683735de35ce None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:08.394227 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe335235-7260-4e69-9fa3-683735de35ce None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:08.451410 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23bcaa-2d04-4fb1-add9-a673a7257ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:08.452085 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23bcaa-2d04-4fb1-add9-a673a7257ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:08.452373 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23bcaa-2d04-4fb1-add9-a673a7257ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:08.452686 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23bcaa-2d04-4fb1-add9-a673a7257ba2 admin admin] PATH_INFO: `/v3/users` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:08.453383 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23bcaa-2d04-4fb1-add9-a673a7257ba2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:08.455449 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23bcaa-2d04-4fb1-add9-a673a7257ba2 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:08.474882 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1436/2884] 10.222.0.43 () {60 vars in 1143 bytes} [Thu Jun 23 22:48:08 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 22:48:08.479068 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b951ebd-5e14-465f-bdb7-5b61109e0c2f admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:08.490183 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b951ebd-5e14-465f-bdb7-5b61109e0c2f admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:08.495046 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1aa45-77cb-4269-802f-7371289ee37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:08.495718 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1aa45-77cb-4269-802f-7371289ee37a admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:08.496005 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1aa45-77cb-4269-802f-7371289ee37a admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:08.496289 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1aa45-77cb-4269-802f-7371289ee37a admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:08.497020 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1aa45-77cb-4269-802f-7371289ee37a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:08.498670 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1aa45-77cb-4269-802f-7371289ee37a admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:08.509974 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1449/2885] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:48:08.514143 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23bcaa-2d04-4fb1-add9-a673a7257ba2 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:08.524220 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23bcaa-2d04-4fb1-add9-a673a7257ba2 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:08.528778 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c924ca-1fae-4725-8fc4-15ba17fb7fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:08.529652 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c924ca-1fae-4725-8fc4-15ba17fb7fee admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:08.529941 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c924ca-1fae-4725-8fc4-15ba17fb7fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:08.530225 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c924ca-1fae-4725-8fc4-15ba17fb7fee admin admin] PATH_INFO: `/v3/projects/2f03ceeefef34eaa8c4aad6c1ef3c2c9/users/c2c7280eea6b4db695fd8c20a332bc2e/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:08.545728 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c924ca-1fae-4725-8fc4-15ba17fb7fee admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f03ceeefef34eaa8c4aad6c1ef3c2c9, user_id=c2c7280eea6b4db695fd8c20a332bc2e, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:08.548965 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c924ca-1fae-4725-8fc4-15ba17fb7fee admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:08.561766 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1437/2886] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:48:08 2022] PUT /identity/v3/projects/2f03ceeefef34eaa8c4aad6c1ef3c2c9/users/c2c7280eea6b4db695fd8c20a332bc2e/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:08.565819 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1aa45-77cb-4269-802f-7371289ee37a admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:08.576083 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1aa45-77cb-4269-802f-7371289ee37a admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:08.632960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3399391-a726-4e01-92d2-b76bd4fc4b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:08.633630 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3399391-a726-4e01-92d2-b76bd4fc4b77 admin admin] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:08.633914 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3399391-a726-4e01-92d2-b76bd4fc4b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:08.634191 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3399391-a726-4e01-92d2-b76bd4fc4b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:08.634911 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3399391-a726-4e01-92d2-b76bd4fc4b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:08.636621 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3399391-a726-4e01-92d2-b76bd4fc4b77 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:08.646976 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1450/2887] 10.222.0.43 () {58 vars in 1121 bytes} [Thu Jun 23 22:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 22:48:08.651908 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c924ca-1fae-4725-8fc4-15ba17fb7fee admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:08.662814 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c924ca-1fae-4725-8fc4-15ba17fb7fee admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:08.668959 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b1240-fd49-4700-9bd7-6f9ee4ac1243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:08.669839 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b1240-fd49-4700-9bd7-6f9ee4ac1243 admin admin] REQUEST_METHOD: `PUT` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:08.670140 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b1240-fd49-4700-9bd7-6f9ee4ac1243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:08.670399 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b1240-fd49-4700-9bd7-6f9ee4ac1243 admin admin] PATH_INFO: `/v3/projects/2f03ceeefef34eaa8c4aad6c1ef3c2c9/users/c2c7280eea6b4db695fd8c20a332bc2e/roles/950c82bd56ab4b4f89ece13a8e6e61f2` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:08.673172 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b1240-fd49-4700-9bd7-6f9ee4ac1243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f03ceeefef34eaa8c4aad6c1ef3c2c9, user_id=c2c7280eea6b4db695fd8c20a332bc2e, role_id=950c82bd56ab4b4f89ece13a8e6e61f2)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:08.675819 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b1240-fd49-4700-9bd7-6f9ee4ac1243 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:08.685967 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1438/2888] 10.222.0.43 () {60 vars in 1482 bytes} [Thu Jun 23 22:48:08 2022] PUT /identity/v3/projects/2f03ceeefef34eaa8c4aad6c1ef3c2c9/users/c2c7280eea6b4db695fd8c20a332bc2e/roles/950c82bd56ab4b4f89ece13a8e6e61f2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:08.697147 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db263749-92ba-4de8-9ab6-8e4101b049ae None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:08.697442 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db263749-92ba-4de8-9ab6-8e4101b049ae None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:08.697729 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db263749-92ba-4de8-9ab6-8e4101b049ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:08.724095 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-db263749-92ba-4de8-9ab6-8e4101b049ae None None] MFA Rules not processed for user `c2c7280eea6b4db695fd8c20a332bc2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:08.725302 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-db263749-92ba-4de8-9ab6-8e4101b049ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:08.794406 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1451/2889] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:48:08 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:08.805901 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b1240-fd49-4700-9bd7-6f9ee4ac1243 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:08.807630 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f98b1240-fd49-4700-9bd7-6f9ee4ac1243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:08.839683 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:08.866884 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b1240-fd49-4700-9bd7-6f9ee4ac1243 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:08.870600 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f4282-7bfc-4070-9234-036e218b7f48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95bc067d93fa4207bec1c85600728085', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:08.871284 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f4282-7bfc-4070-9234-036e218b7f48 service cinder] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:08.871575 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f4282-7bfc-4070-9234-036e218b7f48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:08.871857 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f4282-7bfc-4070-9234-036e218b7f48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:08.929099 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f4282-7bfc-4070-9234-036e218b7f48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:08.930537 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f4282-7bfc-4070-9234-036e218b7f48 service cinder] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:08.940210 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1439/2890] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:48:11.476559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db263749-92ba-4de8-9ab6-8e4101b049ae None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:11.538554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db263749-92ba-4de8-9ab6-8e4101b049ae None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:11.543538 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf8f04-9f1f-4373-8985-a33d17f79d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:11.544427 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf8f04-9f1f-4373-8985-a33d17f79d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:11.544716 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf8f04-9f1f-4373-8985-a33d17f79d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:11.544999 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf8f04-9f1f-4373-8985-a33d17f79d82 admin admin] PATH_INFO: `/v3/users/abf3485a10d64a13acd8f6779f609079` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:11.546220 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bf8f04-9f1f-4373-8985-a33d17f79d82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abf3485a10d64a13acd8f6779f609079)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:11.550002 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bf8f04-9f1f-4373-8985-a33d17f79d82 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:11.555033 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:11.606554 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-a4bf8f04-9f1f-4373-8985-a33d17f79d82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abf3485a10d64a13acd8f6779f609079'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:11.623738 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-a4bf8f04-9f1f-4373-8985-a33d17f79d82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abf3485a10d64a13acd8f6779f609079'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:11.632115 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-a4bf8f04-9f1f-4373-8985-a33d17f79d82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abf3485a10d64a13acd8f6779f609079'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:11.639969 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-a4bf8f04-9f1f-4373-8985-a33d17f79d82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abf3485a10d64a13acd8f6779f609079'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:11.640658 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-a4bf8f04-9f1f-4373-8985-a33d17f79d82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abf3485a10d64a13acd8f6779f609079'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:11.647062 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1452/2891] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:48:11 2022] DELETE /identity/v3/users/abf3485a10d64a13acd8f6779f609079 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:11.651510 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576aebaa-cbaa-4435-b12a-dfe02b244f4d None None] REQUEST_METHOD: `POST` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:11.652211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576aebaa-cbaa-4435-b12a-dfe02b244f4d None None] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:11.652211 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576aebaa-cbaa-4435-b12a-dfe02b244f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:11.670385 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.auth.core [None req-576aebaa-cbaa-4435-b12a-dfe02b244f4d None None] MFA Rules not processed for user `c3c5fce3a5074734811c3407aac7fd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80203) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:48:11.671636 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-576aebaa-cbaa-4435-b12a-dfe02b244f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:11.733267 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1440/2892] 10.222.0.43 () {58 vars in 957 bytes} [Thu Jun 23 22:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:48:11.743568 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf8f04-9f1f-4373-8985-a33d17f79d82 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:11.744974 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-a4bf8f04-9f1f-4373-8985-a33d17f79d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:11.802939 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf8f04-9f1f-4373-8985-a33d17f79d82 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:11.806567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b159d15-7f49-43bc-bab5-c7f3c4465443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37ba527b96634fc2a48b6ca9597d8191', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:11.807256 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b159d15-7f49-43bc-bab5-c7f3c4465443 service neutron] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:11.807543 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b159d15-7f49-43bc-bab5-c7f3c4465443 service neutron] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:11.807859 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b159d15-7f49-43bc-bab5-c7f3c4465443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:11.817284 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b159d15-7f49-43bc-bab5-c7f3c4465443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:11.818723 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b159d15-7f49-43bc-bab5-c7f3c4465443 service neutron] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:11.828266 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1453/2893] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:48:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:48:12.095581 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576aebaa-cbaa-4435-b12a-dfe02b244f4d None None] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:12.097090 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-576aebaa-cbaa-4435-b12a-dfe02b244f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:12.111085 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576aebaa-cbaa-4435-b12a-dfe02b244f4d None None] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:12.116045 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a14de5-8df8-46dc-badf-2ce44fa7e148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:12.116981 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a14de5-8df8-46dc-badf-2ce44fa7e148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:12.117271 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a14de5-8df8-46dc-badf-2ce44fa7e148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:12.117556 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a14de5-8df8-46dc-badf-2ce44fa7e148 admin admin] PATH_INFO: `/v3/projects/41fa6d09534d477f8b69dd5852a23b82` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:12.118812 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a14de5-8df8-46dc-badf-2ce44fa7e148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41fa6d09534d477f8b69dd5852a23b82)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:12.120752 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a14de5-8df8-46dc-badf-2ce44fa7e148 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:12.170104 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-09a14de5-8df8-46dc-badf-2ce44fa7e148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41fa6d09534d477f8b69dd5852a23b82', 'request_id': 'req-09a14de5-8df8-46dc-badf-2ce44fa7e148'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.177674 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-09a14de5-8df8-46dc-badf-2ce44fa7e148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41fa6d09534d477f8b69dd5852a23b82', 'request_id': 'req-09a14de5-8df8-46dc-badf-2ce44fa7e148'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.183419 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-09a14de5-8df8-46dc-badf-2ce44fa7e148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41fa6d09534d477f8b69dd5852a23b82', 'request_id': 'req-09a14de5-8df8-46dc-badf-2ce44fa7e148'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.194513 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-09a14de5-8df8-46dc-badf-2ce44fa7e148 admin admin] The token cache is being invalidate because project 41fa6d09534d477f8b69dd5852a23b82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:48:12.194809 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-09a14de5-8df8-46dc-badf-2ce44fa7e148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.195994 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-09a14de5-8df8-46dc-badf-2ce44fa7e148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.197771 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1441/2894] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:48:12 2022] DELETE /identity/v3/projects/41fa6d09534d477f8b69dd5852a23b82 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:12.201754 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b159d15-7f49-43bc-bab5-c7f3c4465443 service neutron] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:12.213278 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b159d15-7f49-43bc-bab5-c7f3c4465443 service neutron] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:12.272914 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a237e0d-bb6c-46b1-bf22-37ea245ecf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:12.273711 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a237e0d-bb6c-46b1-bf22-37ea245ecf29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:12.274002 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a237e0d-bb6c-46b1-bf22-37ea245ecf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:12.274285 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a237e0d-bb6c-46b1-bf22-37ea245ecf29 admin admin] PATH_INFO: `/v3/users/c2c7280eea6b4db695fd8c20a332bc2e` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:12.275517 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a237e0d-bb6c-46b1-bf22-37ea245ecf29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2c7280eea6b4db695fd8c20a332bc2e)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:12.277470 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a237e0d-bb6c-46b1-bf22-37ea245ecf29 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:12.305663 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:12.328120 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0a237e0d-bb6c-46b1-bf22-37ea245ecf29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2c7280eea6b4db695fd8c20a332bc2e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.342487 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0a237e0d-bb6c-46b1-bf22-37ea245ecf29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2c7280eea6b4db695fd8c20a332bc2e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.351337 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0a237e0d-bb6c-46b1-bf22-37ea245ecf29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2c7280eea6b4db695fd8c20a332bc2e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.358634 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0a237e0d-bb6c-46b1-bf22-37ea245ecf29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2c7280eea6b4db695fd8c20a332bc2e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.359314 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-0a237e0d-bb6c-46b1-bf22-37ea245ecf29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2c7280eea6b4db695fd8c20a332bc2e'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.365642 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1454/2895] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:48:12 2022] DELETE /identity/v3/users/c2c7280eea6b4db695fd8c20a332bc2e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 22:48:12.642787 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a14de5-8df8-46dc-badf-2ce44fa7e148 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:48:12.644345 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-09a14de5-8df8-46dc-badf-2ce44fa7e148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:48:12.655134 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:12.659394 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a14de5-8df8-46dc-badf-2ce44fa7e148 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:48:12.722993 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81578298-a981-4ab1-b319-6670dc21031d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:48:12.723898 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81578298-a981-4ab1-b319-6670dc21031d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:48:12.724185 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81578298-a981-4ab1-b319-6670dc21031d admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:48:12.724468 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81578298-a981-4ab1-b319-6670dc21031d admin admin] PATH_INFO: `/v3/projects/2f03ceeefef34eaa8c4aad6c1ef3c2c9` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:48:12.725672 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81578298-a981-4ab1-b319-6670dc21031d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f03ceeefef34eaa8c4aad6c1ef3c2c9)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:48:12.727645 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81578298-a981-4ab1-b319-6670dc21031d admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:48:12.775104 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-81578298-a981-4ab1-b319-6670dc21031d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f03ceeefef34eaa8c4aad6c1ef3c2c9', 'request_id': 'req-81578298-a981-4ab1-b319-6670dc21031d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.782771 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-81578298-a981-4ab1-b319-6670dc21031d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f03ceeefef34eaa8c4aad6c1ef3c2c9', 'request_id': 'req-81578298-a981-4ab1-b319-6670dc21031d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.788353 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-81578298-a981-4ab1-b319-6670dc21031d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f03ceeefef34eaa8c4aad6c1ef3c2c9', 'request_id': 'req-81578298-a981-4ab1-b319-6670dc21031d'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.800569 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-81578298-a981-4ab1-b319-6670dc21031d admin admin] The token cache is being invalidate because project 2f03ceeefef34eaa8c4aad6c1ef3c2c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:48:12.800872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-81578298-a981-4ab1-b319-6670dc21031d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.801532 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-81578298-a981-4ab1-b319-6670dc21031d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:48:12.803125 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1442/2896] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:48:12 2022] DELETE /identity/v3/projects/2f03ceeefef34eaa8c4aad6c1ef3c2c9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:48:22.361960 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:DELETE=6,keystone:SELECT=19,keystone:INSERT=1 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:48:22.796288 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=49,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:49:03.655428 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d157bd3-e0c0-4414-bbba-0f1050a6f042 None None] REQUEST_METHOD: `POST` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:49:03.655959 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d157bd3-e0c0-4414-bbba-0f1050a6f042 None None] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:49:03.656063 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d157bd3-e0c0-4414-bbba-0f1050a6f042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:49:03.694385 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.auth.core [None req-8d157bd3-e0c0-4414-bbba-0f1050a6f042 None None] MFA Rules not processed for user `df1aeb2be41c49819b811ecfa47d4155`. Rule list: `[]` (Enabled: `True`). {{(pid=80201) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 22:49:03.695616 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8d157bd3-e0c0-4414-bbba-0f1050a6f042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:49:03.775364 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1455/2897] 10.222.0.43 () {58 vars in 967 bytes} [Thu Jun 23 22:49:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 22:49:03.819392 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81578298-a981-4ab1-b319-6670dc21031d admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:49:03.821999 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-81578298-a981-4ab1-b319-6670dc21031d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:49:03.919896 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81578298-a981-4ab1-b319-6670dc21031d admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:49:03.925492 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351a874-cfab-43fd-9b47-f97e4a1d9912 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ce35c43e8294f4cbc880f2219a6eb76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:49:03.926148 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351a874-cfab-43fd-9b47-f97e4a1d9912 service swift] REQUEST_METHOD: `GET` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:49:03.926998 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351a874-cfab-43fd-9b47-f97e4a1d9912 service swift] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:49:03.927303 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351a874-cfab-43fd-9b47-f97e4a1d9912 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:49:03.936786 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351a874-cfab-43fd-9b47-f97e4a1d9912 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:49:03.938796 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351a874-cfab-43fd-9b47-f97e4a1d9912 service swift] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:49:03.947454 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1443/2898] 10.222.0.43 () {60 vars in 1384 bytes} [Thu Jun 23 22:49:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 22:49:10.625975 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d157bd3-e0c0-4414-bbba-0f1050a6f042 None None] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:49:10.628007 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-8d157bd3-e0c0-4414-bbba-0f1050a6f042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:49:10.733040 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d157bd3-e0c0-4414-bbba-0f1050a6f042 None None] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:49:10.738209 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1324ed2-8df9-41cc-8ad2-70612e6d5773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:49:10.739146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1324ed2-8df9-41cc-8ad2-70612e6d5773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:49:10.739437 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1324ed2-8df9-41cc-8ad2-70612e6d5773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:49:10.739715 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1324ed2-8df9-41cc-8ad2-70612e6d5773 admin admin] PATH_INFO: `/v3/users/6867c020bf2a4db3b0a4cebb32c64aa3` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:49:10.741084 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1324ed2-8df9-41cc-8ad2-70612e6d5773 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6867c020bf2a4db3b0a4cebb32c64aa3)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:49:10.742973 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1324ed2-8df9-41cc-8ad2-70612e6d5773 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:49:10.785662 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:49:10.791287 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b1324ed2-8df9-41cc-8ad2-70612e6d5773 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6867c020bf2a4db3b0a4cebb32c64aa3'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:10.804893 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b1324ed2-8df9-41cc-8ad2-70612e6d5773 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6867c020bf2a4db3b0a4cebb32c64aa3'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:10.812748 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b1324ed2-8df9-41cc-8ad2-70612e6d5773 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6867c020bf2a4db3b0a4cebb32c64aa3'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:10.819493 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b1324ed2-8df9-41cc-8ad2-70612e6d5773 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6867c020bf2a4db3b0a4cebb32c64aa3'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:10.820130 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-b1324ed2-8df9-41cc-8ad2-70612e6d5773 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6867c020bf2a4db3b0a4cebb32c64aa3'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:10.825837 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1456/2899] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:49:10 2022] DELETE /identity/v3/users/6867c020bf2a4db3b0a4cebb32c64aa3 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:49:11.008559 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351a874-cfab-43fd-9b47-f97e4a1d9912 service swift] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:49:11.022315 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351a874-cfab-43fd-9b47-f97e4a1d9912 service swift] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:49:11.079378 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef8c84-de51-444c-b8ba-26a638e10d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:49:11.080233 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef8c84-de51-444c-b8ba-26a638e10d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:49:11.080519 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef8c84-de51-444c-b8ba-26a638e10d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:49:11.080798 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef8c84-de51-444c-b8ba-26a638e10d10 admin admin] PATH_INFO: `/v3/projects/cea45368b61d4748a164cc6b76293749` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:49:11.081988 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef8c84-de51-444c-b8ba-26a638e10d10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cea45368b61d4748a164cc6b76293749)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:49:11.084239 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef8c84-de51-444c-b8ba-26a638e10d10 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:49:11.128893 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f4ef8c84-de51-444c-b8ba-26a638e10d10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cea45368b61d4748a164cc6b76293749', 'request_id': 'req-f4ef8c84-de51-444c-b8ba-26a638e10d10'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:11.137697 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f4ef8c84-de51-444c-b8ba-26a638e10d10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cea45368b61d4748a164cc6b76293749', 'request_id': 'req-f4ef8c84-de51-444c-b8ba-26a638e10d10'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:11.144207 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f4ef8c84-de51-444c-b8ba-26a638e10d10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cea45368b61d4748a164cc6b76293749', 'request_id': 'req-f4ef8c84-de51-444c-b8ba-26a638e10d10'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:11.146791 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:49:11.159065 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f4ef8c84-de51-444c-b8ba-26a638e10d10 admin admin] The token cache is being invalidate because project cea45368b61d4748a164cc6b76293749 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:49:11.159355 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f4ef8c84-de51-444c-b8ba-26a638e10d10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:11.160070 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-f4ef8c84-de51-444c-b8ba-26a638e10d10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:11.161606 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1444/2900] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:49:11 2022] DELETE /identity/v3/projects/cea45368b61d4748a164cc6b76293749 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:49:11.165631 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1324ed2-8df9-41cc-8ad2-70612e6d5773 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:49:11.167074 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-b1324ed2-8df9-41cc-8ad2-70612e6d5773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:49:11.180567 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1324ed2-8df9-41cc-8ad2-70612e6d5773 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:49:11.234879 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4d17f-59e4-46ed-b394-8c9a365b6866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:49:11.235771 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4d17f-59e4-46ed-b394-8c9a365b6866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:49:11.236048 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4d17f-59e4-46ed-b394-8c9a365b6866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:49:11.236323 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4d17f-59e4-46ed-b394-8c9a365b6866 admin admin] PATH_INFO: `/v3/users/afbd6e91815a4b4db97ecb90ae9dfef3` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:49:11.237495 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4d17f-59e4-46ed-b394-8c9a365b6866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afbd6e91815a4b4db97ecb90ae9dfef3)` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:49:11.239443 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4d17f-59e4-46ed-b394-8c9a365b6866 admin admin] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:49:11.284146 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c0e4d17f-59e4-46ed-b394-8c9a365b6866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afbd6e91815a4b4db97ecb90ae9dfef3'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:11.295725 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c0e4d17f-59e4-46ed-b394-8c9a365b6866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afbd6e91815a4b4db97ecb90ae9dfef3'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:11.303759 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c0e4d17f-59e4-46ed-b394-8c9a365b6866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afbd6e91815a4b4db97ecb90ae9dfef3'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:11.310559 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c0e4d17f-59e4-46ed-b394-8c9a365b6866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afbd6e91815a4b4db97ecb90ae9dfef3'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:11.311207 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.notifications [None req-c0e4d17f-59e4-46ed-b394-8c9a365b6866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afbd6e91815a4b4db97ecb90ae9dfef3'} {{(pid=80201) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:11.317749 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1457/2901] 10.222.0.43 () {58 vars in 1223 bytes} [Thu Jun 23 22:49:11 2022] DELETE /identity/v3/users/afbd6e91815a4b4db97ecb90ae9dfef3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:49:11.487245 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef8c84-de51-444c-b8ba-26a638e10d10 admin admin] Authenticating user token {{(pid=80203) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:49:11.488992 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.fernet_utils [None req-f4ef8c84-de51-444c-b8ba-26a638e10d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80203) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:49:11.505346 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef8c84-de51-444c-b8ba-26a638e10d10 admin admin] Validating token access rules against request {{(pid=80203) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:49:11.563769 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49976574-d146-4b81-af18-7f819354aeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3c5fce3a5074734811c3407aac7fd6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef1dab34da4006bf2529ee1b5120da', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80203) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:49:11.564587 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49976574-d146-4b81-af18-7f819354aeb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:49:11.564872 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49976574-d146-4b81-af18-7f819354aeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:49:11.565150 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49976574-d146-4b81-af18-7f819354aeb5 admin admin] PATH_INFO: `/v3/projects/19cfa8f118ab4adf804fa31768a7e8cd` {{(pid=80203) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:49:11.566334 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49976574-d146-4b81-af18-7f819354aeb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19cfa8f118ab4adf804fa31768a7e8cd)` {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:49:11.568274 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49976574-d146-4b81-af18-7f819354aeb5 admin admin] RBAC: Authorization granted {{(pid=80203) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:49:11.614089 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-49976574-d146-4b81-af18-7f819354aeb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19cfa8f118ab4adf804fa31768a7e8cd', 'request_id': 'req-49976574-d146-4b81-af18-7f819354aeb5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:11.621256 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-49976574-d146-4b81-af18-7f819354aeb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19cfa8f118ab4adf804fa31768a7e8cd', 'request_id': 'req-49976574-d146-4b81-af18-7f819354aeb5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:11.626751 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-49976574-d146-4b81-af18-7f819354aeb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19cfa8f118ab4adf804fa31768a7e8cd', 'request_id': 'req-49976574-d146-4b81-af18-7f819354aeb5'} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:11.637409 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-49976574-d146-4b81-af18-7f819354aeb5 admin admin] The token cache is being invalidate because project 19cfa8f118ab4adf804fa31768a7e8cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80203) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 22:49:11.637701 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-49976574-d146-4b81-af18-7f819354aeb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:11.638333 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG keystone.notifications [None req-49976574-d146-4b81-af18-7f819354aeb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80203) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 22:49:11.639895 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: [pid: 80203|app: 0|req: 1445/2902] 10.222.0.43 () {58 vars in 1232 bytes} [Thu Jun 23 22:49:11 2022] DELETE /identity/v3/projects/19cfa8f118ab4adf804fa31768a7e8cd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 22:49:21.314321 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=67,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:49:21.634501 dv-cinder-846173-2-iscsi devstack@keystone.service[80203]: DEBUG dbcounter [-] [80203] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80203) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 22:50:06.974688 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4d17f-59e4-46ed-b394-8c9a365b6866 admin admin] Authenticating user token {{(pid=80201) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 22:50:06.976145 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-c0e4d17f-59e4-46ed-b394-8c9a365b6866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:07.068988 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4d17f-59e4-46ed-b394-8c9a365b6866 admin admin] Validating token access rules against request {{(pid=80201) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 22:50:07.128673 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d646a5f-786b-4913-87a3-63a5e6da9137 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101864afd59543e699189917391e6ab1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '56da0975019c44409b53be473131143a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80201) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 22:50:07.129363 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d646a5f-786b-4913-87a3-63a5e6da9137 service placement] REQUEST_METHOD: `GET` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 22:50:07.129651 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d646a5f-786b-4913-87a3-63a5e6da9137 service placement] SCRIPT_NAME: `/identity` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 22:50:07.129931 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d646a5f-786b-4913-87a3-63a5e6da9137 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80201) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 22:50:07.131734 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.fernet_utils [None req-9d646a5f-786b-4913-87a3-63a5e6da9137 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80201) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 22:50:07.143444 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d646a5f-786b-4913-87a3-63a5e6da9137 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 22:50:07.145721 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d646a5f-786b-4913-87a3-63a5e6da9137 service placement] RBAC: Authorization granted {{(pid=80201) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 22:50:07.156426 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: [pid: 80201|app: 0|req: 1458/2903] 10.222.0.43 () {60 vars in 1365 bytes} [Thu Jun 23 22:50:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 22:50:17.151299 dv-cinder-846173-2-iscsi devstack@keystone.service[80201]: DEBUG dbcounter [-] [80201] Writing DB stats keystone:SELECT=67 {{(pid=80201) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}